Skip to content
Merged
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
fixed computation of max value bit width in patched base encoding
  • Loading branch information
Baptiste Mehat committed Dec 11, 2025
commit 3974efa99307b8a608abbbc79f3d0a8fdece8840
2 changes: 1 addition & 1 deletion src/encoding/integer/rle_v2/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -510,7 +510,7 @@ fn determine_variable_run_encoding<N: NInt, S: EncodingSign>(
}

// Aka 100th percentile
let base_reduced_literals_max_bit_width = max_data_value.closest_aligned_bit_width();
let base_reduced_literals_max_bit_width = max_data_value.bits_used();
// 95th percentile width is used to find the 5% of values to encode with patches
let base_reduced_literals_95th_percentile_bit_width =
calculate_percentile_bits(&base_reduced_literals, 0.95);
Expand Down