Skip to content
Closed
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
SPARK-20213: Throw errors during testing for nested execution.
  • Loading branch information
rdblue committed May 2, 2017
commit 6ad635e952da52957a9609172aca827d971071f8
Original file line number Diff line number Diff line change
Expand Up @@ -116,11 +116,16 @@ object SQLExecution extends Logging {
// Some operations will start nested executions. For example, CacheTableCommand will uses
// Dataset#count to materialize cached records when caching is not lazy. Because there are
// legitimate reasons to nest executions in withNewExecutionId, this logs a warning but does
// not throw an exception to avoid failing at runtime.
// not throw an exception to avoid failing at runtime. Exceptions will be thrown for tests
// to ensure that nesting is avoided.
//
// To avoid this warning, use nestedExecution { ... }
// To avoid this warning, use nested { ... }
if (!Option(sc.getLocalProperty(ALLOW_NESTED_EXECUTION)).exists(_.toBoolean)) {
logWarning(s"$EXECUTION_ID_KEY is already set")
if (testing) {
logWarning(s"$EXECUTION_ID_KEY is already set")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

According to the comment, we should throw the exception here and log warning at runtime?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch. Fixed.

} else {
throw new IllegalStateException(s"$EXECUTION_ID_KEY is already set")
}
}
body
}
Expand Down