Skip to content

[SPARK-24913][SQL] Make AssertNotNull and AssertTrue non-deterministic #22947

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
mgaido91 wants to merge 6 commits into from
Closed

[SPARK-24913][SQL] Make AssertNotNull and AssertTrue non-deterministic #22947

Changes from 1 commit
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
332db30
[SPARK-24913][SQL] Make AssertNotNull and AssertTrue non-deterministic
mgaido91 Nov 5, 2018
895e095
fix test failures
mgaido91 Nov 5, 2018
4bd7c2d
fix test failures
mgaido91 Nov 5, 2018
0a02dd9
Revert "fix test failures"
mgaido91 Nov 7, 2018
129d4f7
Revert "fix test failures"
mgaido91 Nov 7, 2018
1caa284
introduce hasSideEffect
mgaido91 Nov 7, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Revert "fix test failures" 
This reverts commit 4bd7c2d.
  • Loading branch information
@mgaido91
mgaido91 committed Nov 7, 2018
commit 0a02dd99ac30d49bb95f737112fc803006ec0efb
6 changes: 3 additions & 3 deletions sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/CheckAnalysis.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -368,11 +368,11 @@ trait CheckAnalysis extends PredicateHelper {
case o if o.expressions.exists(!_.deterministic) &&
!o.isInstanceOf[Project] && !o.isInstanceOf[Filter] &&
!o.isInstanceOf[Aggregate] && !o.isInstanceOf[Window] &&
!o.isInstanceOf[DeserializeToObject] && !o.isInstanceOf[SerializeFromObject] =>
!o.isInstanceOf[DeserializeToObject] =>
// The rule above is used to check Aggregate operator.
failAnalysis(
s"""nondeterministic expressions are only allowed in Project, Filter,
| Aggregate, Window, SerializeFromObject or DeserializeToObject, found:
s"""nondeterministic expressions are only allowed in
|Project, Filter, Aggregate, Window or DeserializeToObject, found:
| ${o.expressions.map(_.sql).mkString(",")}
|in operator ${operator.simpleString}
""".stripMargin)
Expand Down