-
Notifications
You must be signed in to change notification settings - Fork 29k
[SPARK-4391][SQL] Configure parquet filters using SQLConf #3258
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
Test build #23338 has started for PR 3258 at commit
|
|
Test build #23338 has finished for PR 3258 at commit
|
|
Test FAILed. |
|
Test build #23340 has started for PR 3258 at commit
|
|
Test build #23340 has finished for PR 3258 at commit
|
|
Test PASSed. |
|
Test build #23395 has started for PR 3258 at commit
|
This is more uniform with the rest of SQL configuration and allows it to be turned on and off without restarting the SparkContext. In this PR I also turn off filter pushdown by default due to a number of outstanding issues (in particular SPARK-4258). When those are fixed we should turn it back on by default. Author: Michael Armbrust <[email protected]> Closes #3258 from marmbrus/parquetFilters and squashes the following commits: 5655bfe [Michael Armbrust] Remove extra line. 15e9a98 [Michael Armbrust] Enable filters for tests 75afd39 [Michael Armbrust] Fix comments 78fa02d [Michael Armbrust] off by default e7f9e16 [Michael Armbrust] First draft of correctly configuring parquet filter pushdown (cherry picked from commit e47c387) Signed-off-by: Michael Armbrust <[email protected]>
|
Test build #23395 has finished for PR 3258 at commit
|
|
Test PASSed. |
This is more uniform with the rest of SQL configuration and allows it to be turned on and off without restarting the SparkContext. In this PR I also turn off filter pushdown by default due to a number of outstanding issues (in particular SPARK-4258). When those are fixed we should turn it back on by default.