Skip to content

[SPARK-34538][SQL] Hive Metastore support filter by not-in #31646

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
ulysses-you wants to merge 15 commits into from
Closed

[SPARK-34538][SQL] Hive Metastore support filter by not-in #31646

Changes from 1 commit
Commits
Show all changes
15 commits
Select commit Hold shift + click to select a range
3f95df6
init
ulysses-you Feb 25, 2021
3cefafe
fix test
ulysses-you Feb 26, 2021
0dc1c4a
add null test
ulysses-you Feb 26, 2021
b9c7f44
avoid metastore stack over flow
ulysses-you Feb 26, 2021
cf3ba56
nit
ulysses-you Feb 26, 2021
0415448
Merge branch 'master' of https://github.com/apache/spark into SPARK-3…
ulysses-you Feb 26, 2021
113bd62
doc
ulysses-you Mar 10, 2021
01b7466
add null precheck
ulysses-you Mar 10, 2021
e37f5f9
test
ulysses-you Mar 10, 2021
93eedd3
pattern
ulysses-you Mar 10, 2021
e8c7b6c
move forward
ulysses-you Mar 10, 2021
062b1d4
doc
ulysses-you Mar 10, 2021
4704ec3
not in datetype suite
ulysses-you Mar 10, 2021
5acbf6a
HivePartitionFilteringSuite
ulysses-you Mar 11, 2021
17aea47
remove int test
ulysses-you Mar 11, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
doc
  • Loading branch information
@ulysses-you
ulysses-you committed Mar 10, 2021
commit 062b1d4d96256c4c05a00debf50aac8d3e64f614
4 changes: 2 additions & 2 deletions sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -864,8 +864,8 @@ object SQLConf {
"Metastore. When the set size exceeds the threshold, we rewrite the InSet predicate " +
"to be greater than or equal to the minimum value in set and less than or equal to the " +
"maximum value in set. Larger values may cause Hive Metastore stack overflow. But for " +
"the predicate of Not InSet which values exceeds the threshold, we won't push it to " +
"Hive Metastore.")
"InSet inside Not with values exceeding the threshold, we won't push it to Hive Metastore."
)
.version("3.1.0")
.internal()
.intConf
Expand Down