Skip to content

[SPARK-21247][SQL] Type comparison should respect case-sensitive SQL conf #18460

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
dongjoon-hyun wants to merge 9 commits into from
Closed

[SPARK-21247][SQL] Type comparison should respect case-sensitive SQL conf #18460

Changes from 1 commit
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
037120b
[SPARK-21247][SQL] Allow case-insensitive type equality in Set operation
dongjoon-hyun Jun 28, 2017
cded719
Add comments.
dongjoon-hyun Jul 7, 2017
180e1b3
Address comments.
dongjoon-hyun Jul 8, 2017
2343d97
Add test case.
dongjoon-hyun Jul 8, 2017
8fa884e
Move to findTightestCommonType.
dongjoon-hyun Jul 30, 2017
b2d1338
Move testcase, too.
dongjoon-hyun Jul 30, 2017
c72aa18
Add test cases.
dongjoon-hyun Oct 6, 2017
67a037c
Update test cases.
dongjoon-hyun Oct 6, 2017
52d19d3
Address comments.
dongjoon-hyun Oct 11, 2017
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Add comments.
  • Loading branch information
@dongjoon-hyun
dongjoon-hyun committed Sep 8, 2017
commit cded71940a7918bdb155e34d06af0cccf9c3362b
4 changes: 4 additions & 0 deletions sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/TypeCoercion.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -148,6 +148,10 @@ object TypeCoercion {
findWiderTypeForTwo(et1, et2).map(ArrayType(_, containsNull1 || containsNull2))
case (st1 @ StructType(fields1), st2 @ StructType(fields2)) if st1.sameType(st2) =>
Some(StructType(fields1.zip(fields2).map { case (sf1, sf2) =>
// Since `st1.sameType(st2)` is true, two StructTypes have the same DataType
// except `name` (in case of `spark.sql.caseSensitive=false`) and `nullable`.
// - Different names: use a lower case name because findWiderTypeForTwo is commutative.
// - Different nullabilities: `nullable` is true iff one of them is nullable.
val name = if (sf1.name == sf2.name) sf1.name else sf1.name.toLowerCase(Locale.ROOT)
val dataType = findWiderTypeForTwo(sf1.dataType, sf2.dataType).get
StructField(name, dataType, nullable = sf1.nullable || sf2.nullable)
Expand Down