Skip to content

[SPARK-35298][SQL] Migrate to transformWithPruning for rules in Optimizer.scala #32439

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
sigmod wants to merge 12 commits into from
Closed

[SPARK-35298][SQL] Migrate to transformWithPruning for rules in Optimizer.scala #32439

Changes from 1 commit
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
6a35a88
update
sigmod May 3, 2021
c9d0ef0
update
sigmod May 4, 2021
7fbd213
merge master
sigmod May 4, 2021
98830eb
support pruning in more rules
sigmod May 5, 2021
7a6b879
update
sigmod May 5, 2021
381e38f
fix test failure
sigmod May 5, 2021
564d8fb
update
sigmod May 5, 2021
f376055
merge master
sigmod May 6, 2021
27e92ea
merge master
sigmod May 11, 2021
0df10d1
Address Gengliang's comment
sigmod May 11, 2021
2408a18
Merge branch 'master' into optimizer
sigmod May 11, 2021
5027ebc
Merge branch 'master' into optimizer
sigmod May 12, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
update
  • Loading branch information
@sigmod
sigmod committed May 5, 2021
commit 564d8fb2189f9bfc6db658f312dc593442e07223
2 changes: 1 addition & 1 deletion sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2053,7 +2053,7 @@ object OptimizeLimitZero extends Rule[LogicalPlan] {
LocalRelation(plan.output, data = Seq.empty, isStreaming = plan.isStreaming)

def apply(plan: LogicalPlan): LogicalPlan = plan.transformUpWithPruning(
_.containsPattern(LIMIT)) {
_.containsAllPatterns(LIMIT, LITERAL)) {
// Nodes below GlobalLimit or LocalLimit can be pruned if the limit value is zero (0).
// Any subtree in the logical plan that has GlobalLimit 0 or LocalLimit 0 as its root is
// semantically equivalent to an empty relation.
Expand Down