Skip to content

[SPARK-20213][SQL][UI] Fix DataFrameWriter operations in SQL UI tab. #17540

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
rdblue wants to merge 15 commits into from
Closed

[SPARK-20213][SQL][UI] Fix DataFrameWriter operations in SQL UI tab. #17540

Changes from 1 commit
Commits
Show all changes
15 commits
Select commit Hold shift + click to select a range
ffb3e84
SPARK-20213: Fix DataFrameWriter operations in SQL UI tab.
rdblue Apr 5, 2017
a59872e
SPARK-20213: Add check to ensure exection ID is set.
rdblue Apr 6, 2017
82f9c36
SPARK-20213: Add withNewExecutionId to commands run by spark.sql.
rdblue Apr 6, 2017
7cf0f3c
SPARK-20213: Allow nested withNewExecutionId.
rdblue Apr 10, 2017
2afefa9
SPARK-20213: Use withAction instead of withNewExecutionId.
rdblue Apr 10, 2017
6ad635e
SPARK-20213: Throw errors during testing for nested execution.
rdblue Apr 10, 2017
163a07a
SPARK-20213: Add withNewExecutionId to test method.
rdblue Apr 11, 2017
653cdf3
SPARK-20213: Fix failing tests.
rdblue Apr 12, 2017
85b819f
SPARK-20213: Call withNewExecutionId from StreamExecution.
rdblue Apr 12, 2017
a043e5c
SPARK-20213: Fix SQLListener test.
rdblue Apr 14, 2017
20a0796
SPARK-20213: Remove SPARK-10548 repro test.
rdblue Apr 14, 2017
aad22f8
SPARK-20213: Fix pyspark test failure.
rdblue Apr 20, 2017
6752ff2
SPARK-20213: Remove metrics work-around.
rdblue Apr 30, 2017
90045cf
SPARK-20213: Fix backward logic for nesting.
rdblue May 2, 2017
f63b773
SPARK-20213: Fix more tests with nested SQL executions.
rdblue May 2, 2017
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
SPARK-20213: Fix SQLListener test. 
Opening up metrics in the last commit caused this test to fail.
  • Loading branch information
@rdblue
rdblue committed May 2, 2017
commit a043e5c51a96f31d96691a1431d5f6a266b1734c
4 changes: 3 additions & 1 deletion sql/core/src/test/scala/org/apache/spark/sql/execution/ui/SQLListenerSuite.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -92,7 +92,9 @@ class SQLListenerSuite extends SparkFunSuite with SharedSQLContext with JsonTest

test("basic") {
def checkAnswer(actual: Map[Long, String], expected: Map[Long, Long]): Unit = {
assert(actual.size == expected.size)
// TODO: Remove greater-than case when all metrics are correctly linked into the physical plan
// See SQLListener#getExecutionMetrics
assert(actual.size >= expected.size)
expected.foreach { e =>
// The values in actual can be SQL metrics meaning that they contain additional formatting
// when converted to string. Verify that they start with the expected value.
Expand Down