ClickHouse performance comparison

Tested Commits ?

ReferencePatched
ClickHouse local version 25.7.1.291 (official build).

commit 896ccc4536c18d89a3ce2454acc085ea78bf0eb1
Author: Vladimir Cherkasov 
Date:   Tue Jun 17 18:29:57 2025 +0000

    Merge pull request #80165 from ClickHouse/vidmir/issue_77369
    
    Fix logical error with <=> operator and Join storage

Changes in Performance ?

Old, sNew, sRatio of speedup (-) or slowdown (+)Relative difference (new − old) / oldp < 0.01 thresholdTest#Query
0.3160.351+1.11x0.110.11array_index_low_cardinality_strings3SELECT count() FROM perf_lc_str WHERE indexOf(arr, 'asdf337') > 0
0.1760.186+1.056x0.0550.055count_from_formats22SELECT count() FROM table_Avro FORMAT Null
0.1730.182+1.054x0.0540.054count_from_formats51SELECT count() FROM table_Avro group by _file, _path FORMAT Null

Unstable Queries ?

Old, sNew, sRelative difference (new - old)/oldp < 0.01 thresholdTest#Query
0.4210.6210.4740.481general_purpose_hashes11SELECT count() from numbers_mt(500000000) where not ignore(metroHash64(toString(1000000000+number)))
0.2550.2610.0240.341file_table_function21 INSERT INTO FUNCTION file('test_file', 'JSONCompactEachRowWithNamesAndTypes', 'key UInt64, value1 UInt64, value2 UInt64, value3 UInt64, value4 UInt64, value5 UInt64') SELECT number, number, number, number, number, number FROM numbers(1000000)
0.1320.140.0610.234jit_aggregate_functions309 SELECT groupBitOrIf(value_1, predicate), groupBitOrIf(value_2, predicate), sumIf(toUInt256(value_3), predicate), groupBitOrIf(value_3, predicate), groupBitOrIf(value_4, predicate), groupBitOrIf(value_5, predicate) FROM jit_test_merge_tree GROUP BY key FORMAT Null

Test Performance Changes ?

TestRatio of speedup (-) or slowdown (+)QueriesTotal not OKChanged perfUnstable
Total-1.003x608936
empty_string_deserialization-1.306x1000
jit_large_requests-1.069x2000
parallel_insert-1.069x1000
aggregation_in_order_2-1.067x9000
memory_bound_merging-1.046x1000
local_replica-1.038x1000
formats_columns_sampling-1.038x10000
reading_from_file-1.036x1000
jit_aggregate_functions-1.034x10404
lazyMaterialization-1.023x2000
group_array_moving_sum-1.02x10000
bloom_filter_insert-1.02x1000
point_in_polygon_3d_huge_multipolygon-1.019x1000
set_index_analysis-1.018x1000
decimal_aggregates-1.018x10000
uniqExactIf-1.016x5000
bit_operations_fixed_string_numbers-1.013x2000
ip_trie-1.012x4000
new_json_type-1.012x3000
arrayAUCPR-1.012x1000
prewhere-1.01x1000
compact_part_subcolumns-1.01x2000
coalesce-1.009x1000
redundant_functions_in_order_by-1.009x3000
join_max_streams-1.008x3000
optimized_select_final_one_part-1.006x1000
merge_tree_insert+1.006x10000
file_table_function+1.006x10101
decimal_casts+1.007x10000
information_value+1.008x2000
low_cardinality_query+1.008x2000
memory_cache_friendliness+1.009x1000
parallel_hash_build_phase+1.009x1000
writing_valid_utf8+1.009x3000
url_hits+1.009x9000
join_convert_outer_to_inner+1.009x5000
trivial_insert_select_from_files+1.01x9000
optimize_window_funnel+1.01x1000
count_from_formats+1.011x10220
join_append_block+1.011x2000
agg_functions_argmin_argmax+1.011x10000
codecs_int_insert+1.011x10000
array_reduce+1.012x6000
codec_none+1.013x1000
general_purpose_hashes+1.015x10101
string_set+1.016x7000
function_tokens+1.018x3000
re2_regex_caching+1.021x7000
codecs_float_insert+1.023x10000
array_index_low_cardinality_strings+1.04x10110
quantile_merge+1.234x1000

Test Times ?

TestWall clock time, entire test, sTotal client time for measured query runs, sQueriesLongest query, total for measured runs, sAverage query wall clock time, sShortest query, total for measured runs, s
Total5336.9192990.6760824.7448.7781.851
codec_none33.3252.02812.02833.3252.028
read_in_order_many_parts248.06423.70887.37131.0082.025
uniq_with_key161.70654.731711.75423.1016.072
read_from_comp_parts113.05612.53553.14122.6112.055
aggregation_in_order_2194.84247.191910.36721.6492.009
alter_select83.4325.566410.86720.8572.013
merge_tree_many_partitions20.2712.02312.02320.2712.023
function_tokens59.96144.514324.74419.9872.124
parallel_hash_build_phase19.32914.909114.90919.32914.909
parallel_final189.80420.24102.06718.982.006
count_from_formats185.30123.516103.7618.532.011
string_bytes_functions36.72125.445223.41818.362.027
group_array_moving_sum181.278111.631018.67218.1285.446
compact_part_subcolumns35.0464.17422.08817.5232.087
coalesce17.11314.418114.41817.11314.418
array_index_low_cardinality_strings170.46584.4551011.86217.0474.003
window_functions_downstream_multithreading95.58174.492624.13415.932.102
optimized_select_final_one_part15.7572.05712.05715.7572.057
uniq_without_key_dist69.57320.3657.19713.9152.236
scalar13.619.33619.33613.619.336
distinct_in_order103.93516.62482.21412.9922.01
quantile_merge12.912.08612.08612.912.086
join_convert_outer_to_inner63.37542.619518.62912.6752.009
file_table_function125.354106.8161017.44812.5352.017
agg_functions_argmin_argmax124.952104.1931016.84612.4952.016
group_by_fixed_keys123.84629.835105.13812.3852.046
point_in_polygon_3d_huge_multipolygon12.28510.154110.15412.28510.154
re2_regex_caching83.71146.53719.18111.9592.014
parallel_insert11.939.31519.31511.939.315
codecs_float_insert119.192102.661018.08211.9192.6
url_hits106.74483.883912.74411.862.887
unary_logical_functions115.717100.971016.48211.5726.354
normalize_utf845.83139.628410.32511.4589.558
hierarchical_dictionaries114.35898.2291017.37911.4362.073
prewhere11.0719.16619.16611.0719.166
trivial_insert_select_from_files96.03452.85398.79810.674.744
array_fill63.426.01969.88910.5672.058
codecs_int_insert97.97684.6061017.2989.7982.142
hashed_dictionary_load_factor97.46477.2861016.2029.7462.003
empty_string_deserialization9.3033.51413.5149.3033.514
empty_string_serialization9.1037.36817.3689.1037.368
round_down45.30839.187513.8939.0622.067
final_with_lonely_parts18.1035.87823.6269.0512.251
replace_int25643.97237.42259.2548.7943.53
has_all86.04629.361105.9398.6052.012
set_hits51.08644.031616.0388.5142.257
random_string_utf859.16750.78879.7658.4524.332
join_append_block15.75313.17226.9767.8766.196
memory_bound_merging7.6885.57115.5717.6885.571
norm_distance72.13753.53108.1347.2143.556
function_calculation_after_sorting_and_limit21.34918.112313.8917.1162.022
inserts_arrays_lowcardinality14.10610.26628.2357.0532.031
memory_cache_friendliness6.7162.35312.3536.7162.353
reading_from_file6.585.1715.176.585.17
optimize_functions_to_subcolumns39.42312.08462.026.572.008
jit_aggregate_functions65.66843.223106.9126.5672.032
select_format65.07755.9591014.8646.5082.005
group_array_sorted62.1620.466102.136.2162.003
ip_trie24.4549.27843.1036.1142.043
prewhere_with_row_level_filter30.48310.71952.246.0972.073
if_to_multiif29.18325.332515.9455.8372.018
json_extract_rapidjson57.89148.911109.1945.7892.051
decimal_casts56.19639.366106.6995.622.045
leftpad38.56531.87176.8675.5092.036
uniqExactIf27.35314.42356.295.4712.009
local_replica5.4214.22714.2275.4214.227
merge_tree_insert53.66645.0021014.2885.3672.01
formats_columns_sampling53.62227.845105.8445.3622.013
hashed_dictionary50.81527.676109.1865.0812.029
cpu_synthetic50.4343.3781012.4265.0432.037
general_purpose_hashes49.47942.9041014.4574.9482.041
string_set34.02424.59675.9674.8612.021
consistent_hashes28.97125.434610.4774.8292.006
order_by_single_column32.70828.29279.064.6732.025
monotonous_order_by8.9017.47425.4594.452.015
array_reduce26.68513.88362.4064.4482.183
sum44.32227.382107.8514.4322.023
decimal_aggregates42.40632.137107.154.2412.033
bloom_filter_insert4.183.01513.0154.183.015
insert_select_squashing3.9012.57712.5773.9012.577
short_circuit_functions38.2732.748107.2143.8271.851
new_json_type11.3099.07334.1953.772.081
json_type11.0968.86634.2853.6992.226
orc_tuple_field_prune11.0246.21132.1613.6752.017
subqueries7.2035.74122.8723.6012.869
date_time_long35.01830.283107.1413.5022.07
lazyMaterialization6.9445.03122.6493.4722.383
set_index_analysis3.3512.01412.0143.3512.014
arrayAUCPR3.3392.37912.3793.3392.379
date_time_short32.08727.844107.0013.2092.005
trim_numbers18.95616.30964.2873.1592.09
optimize_window_funnel3.142.02812.0283.142.028
bloom_filter_select6.2144.03622.0213.1072.014
point_in_polygon_const_3d3.0242.08812.0883.0242.088
explain_ast2.9962.06712.0672.9962.067
h32.9872.20612.2062.9872.206
join_max_streams8.9387.73733.7112.9792.012
array_index_low_cardinality_numbers26.18521.97192.8212.9092.012
column_array_filter23.17319.0682.9882.8972.008
complex_array_creation5.7724.49822.4612.8862.037
prepare_hash_before_merge11.2138.40742.2022.8032.031
orc_filter_push_down5.554.02422.0152.7752.008
sumIf2.7512.07212.0722.7512.072
jit_large_requests5.3894.16122.1222.6942.04
analyze_array_tuples2.6942.02512.0252.6942.025
redundant_functions_in_order_by7.9336.56932.412.6442.051
low_cardinality_query5.2614.02822.0152.6312.013
aggregate_functions_of_group_by_keys5.2394.26222.1932.622.069
range5.2174.2222.1212.6092.099
rewrite_array_exists5.1644.09722.0622.5822.035
bit_operations_fixed_string_numbers5.1444.05622.0412.5722.015
writing_valid_utf87.6986.31732.1622.5662.068
sort_radix_trivial7.4956.3532.1222.4982.114
information_value4.9654.08122.0762.4832.005
bigint_formatting9.9068.50742.1592.4772.097
group_by_consecutive_keys14.14512.38762.1472.3572.022
countIf6.9926.13232.0682.3312.004
logical_functions_large23.1320.909102.5242.3132.007
sparse_grams4.4643.84521.9792.2321.866
countDigits8.8268.05542.0162.2062.01

Metric Changes ?

MetricOld median valueNew median valueRelative differenceTimes difference
jemalloc.retained40913180672444601917440.0871.087
MemoryDataAndStack48917524480528963461120.0811.081
MemoryVirtual51924414464559032401920.0771.077
KeeperOpenFileDescriptorCount535499-0.0671.072
jemalloc.arenas.all.pdirty23395.519642.5-0.161.191

All queries Log Test output