Skip to content

[SPARK-13484][SQL] Prevent illegal NULL propagation when filtering outer-join results #11371

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
maropu wants to merge 7 commits into from
Closed

[SPARK-13484][SQL] Prevent illegal NULL propagation when filtering outer-join results #11371

Changes from 1 commit
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
1723a1a
Avoid illegal NULL propagation
maropu Feb 25, 2016
bc3a3a7
Add comments
maropu Feb 26, 2016
b56da9f
Add a new rule to solve illegal references
maropu Feb 27, 2016
274c542
Use foreach not map
maropu Mar 2, 2016
4a7121e
Solve illegal references in Projects
maropu Apr 15, 2016
a4903b6
Add tests in DataFrameJoinSuite
maropu Apr 15, 2016
bd13652
Fix test codes in ResolveNaturalJoinSuite
maropu Apr 15, 2016
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Use foreach not map
  • Loading branch information
@maropu
maropu committed May 12, 2016
commit 274c542be7d0019afe705d517d09cf3fe666389f
2 changes: 1 addition & 1 deletion sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1473,7 +1473,7 @@ class Analyzer(
q.transform {
case f @ Filter(filterCondition, j @ Join(_, _, _, _)) =>
val joinOutput = new ArrayBuffer[(Attribute, Attribute)]
j.output.map {
j.output.foreach {
case a: AttributeReference => joinOutput += ((a, a))
}
val joinOutputMap = AttributeMap(joinOutput)
Expand Down