Skip to content

[Config] [Spark] Enable fastInterleaveBits by default #5815

@felipepessoto

Description

@felipepessoto

Which Delta project/connector is this regarding?

  • Spark
  • Standalone
  • Flink
  • Kernel
  • Other (fill in here)

Describe the problem

The plan was to enable it by default after a couple releases (after 2.0/2.1), but I think we completely forgot it.

Given this is a new approach, should we set the default to false and make it default to true after the couple of releases and more confidence? - #1160 (comment)

The PR says perf is around 30% better: #1160

@vkorukanti, @cfmcgrady

Environment information

  • Delta Lake version: 4.0
  • Spark version: 4.0
  • Scala version: 2.13

Willingness to contribute

The Delta Lake Community encourages bug fix contributions. Would you or another member of your organization be willing to contribute a fix for this bug to the Delta Lake code base?

  • Yes. I can contribute a fix for this bug independently.
  • Yes. I would be willing to contribute a fix for this bug with guidance from the Delta Lake community.
  • No. I cannot contribute a bug fix at this time.

Metadata

Metadata

Assignees

Labels

bugSomething isn't workingdelta-sparkIssues and PRs related to general Delta-Spark issues

Type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions