Skip to content

[SPARK-34538][SQL] Hive Metastore support filter by not-in #31646

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
ulysses-you wants to merge 15 commits into from
Closed

[SPARK-34538][SQL] Hive Metastore support filter by not-in #31646

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
15 commits
Select commit Hold shift + click to select a range
3f95df6
init
ulysses-you Feb 25, 2021
3cefafe
fix test
ulysses-you Feb 26, 2021
0dc1c4a
add null test
ulysses-you Feb 26, 2021
b9c7f44
avoid metastore stack over flow
ulysses-you Feb 26, 2021
cf3ba56
nit
ulysses-you Feb 26, 2021
0415448
Merge branch 'master' of https://github.com/apache/spark into SPARK-3…
ulysses-you Feb 26, 2021
113bd62
doc
ulysses-you Mar 10, 2021
01b7466
add null precheck
ulysses-you Mar 10, 2021
e37f5f9
test
ulysses-you Mar 10, 2021
93eedd3
pattern
ulysses-you Mar 10, 2021
e8c7b6c
move forward
ulysses-you Mar 10, 2021
062b1d4
doc
ulysses-you Mar 10, 2021
4704ec3
not in datetype suite
ulysses-you Mar 10, 2021
5acbf6a
HivePartitionFilteringSuite
ulysses-you Mar 11, 2021
17aea47
remove int test
ulysses-you Mar 11, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
fix test
  • Loading branch information
@ulysses-you
ulysses-you committed Feb 26, 2021
commit 3cefafebb11f04011aa18e394dbaf01a14e182a3
2 changes: 1 addition & 1 deletion sql/hive/src/test/scala/org/apache/spark/sql/hive/client/FiltersSuite.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -128,7 +128,7 @@ class FiltersSuite extends SparkFunSuite with Logging with PlanTest {
(Not(InSet(a("datecol", DateType),
Set(Literal(Date.valueOf("2020-01-01")).eval(),
Literal(Date.valueOf("2020-01-02")).eval())))) :: Nil,
"""(datecol != "2020-01-01" and datecol != "2020-01-02")""")
"""(datecol != 2020-01-01 and datecol != 2020-01-02)""")

// Applying the predicate `x IN (NULL)` should return an empty set, but since this optimization
// will be applied by Catalyst, this filter converter does not need to account for this.
Expand Down
2 changes: 1 addition & 1 deletion sql/hive/src/test/scala/org/apache/spark/sql/hive/client/HivePartitionFilteringSuite.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -448,7 +448,7 @@ class HivePartitionFilteringSuite(version: String)
dsValue,
hValue,
chunkValue,
Seq("2019-01-03") ++ defaultPartition,
Seq("2019-01-03"),
dateStrValue
)
}
Expand Down