Skip to content

[SPARK-21417][SQL] Infer join conditions using propagated constraints #18692

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
aokolnychyi wants to merge 5 commits into from
Closed

[SPARK-21417][SQL] Infer join conditions using propagated constraints #18692

Changes from 1 commit
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
e0e6ad3
[SPARK-21417][SQL] Infer join conditions using propagated constraints
Jul 18, 2017
b69185c
[SPARK-21417][SQL] Added a comment
Oct 18, 2017
3e090f9
Consider only CROSS joins without conditions & switch to a map-based …
Nov 24, 2017
0e5a9f2
separate EquivalentExpressionMap
Nov 30, 2017
9ab91a1
Fix nonascii.message
Nov 30, 2017
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
[SPARK-21417][SQL] Added a comment
  • Loading branch information
aokolnychyi committed Oct 18, 2017
commit b69185c2a7466b69a3f244a257449dbf1dd0ee21
4 changes: 3 additions & 1 deletion sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/joins.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -155,7 +155,9 @@ object EliminateOuterJoin extends Rule[LogicalPlan] with PredicateHelper {

/**
* A rule that uses propagated constraints to infer join conditions. The optimization is applicable
* only to CROSS joins.
* only to CROSS joins. For other join types, adding inferred join conditions would potentially
* shuffle children as child node's partitioning won't satisfy the JOIN node's requirements
* which otherwise could have.
*
* For instance, if there is a CROSS join, where the left relation has 'a = 1' and the right
* relation has 'b = 1', then the rule infers 'a = b' as a join predicate.
Expand Down