Skip to content

Use branchless index clamping and add get_batch_direct to RleDecoder #9581

@Dandandan

Description

@Dandandan

Description

The RLE dictionary decoding path uses if/else branching to select between checked and unchecked indexing. This can be replaced with a single branchless .min(max_idx) clamp that:

  • Prevents UB on corrupt parquet files (indices clamped to valid range)
  • Removes the if/else branch, simplifying codegen
  • Improves i32 dict perf by ~13%

Additionally, adding a get_batch_direct method to RleDecoder that exposes RLE vs bit-packed batches via a callback allows callers to handle each case optimally (e.g., using repeat_n for RLE runs).

🤖 Generated with Claude Code

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions