Skip to content

[SPARK-18657][SPARK-18668] Make StreamingQuery.id persists across restart and not auto-generate StreamingQuery.name #16113

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
tdas wants to merge 15 commits into from
Closed

[SPARK-18657][SPARK-18668] Make StreamingQuery.id persists across restart and not auto-generate StreamingQuery.name #16113

Changes from 1 commit
Commits
Show all changes
15 commits
Select commit Hold shift + click to select a range
26f3cf9
Made StreamingQuery.id persist across restart, added StreamingQuery.r…
tdas Dec 2, 2016
f103def
Fixed test
tdas Dec 2, 2016
f55b852
Made codahale metrics use id instead of name
tdas Dec 2, 2016
c20f4fe
Made name to be default null
tdas Dec 2, 2016
aca547f
Merge remote-tracking branch 'apache-github/master' into HEAD
tdas Dec 2, 2016
bec2fb3
Some more changes
tdas Dec 2, 2016
0554e5e
Added tests
tdas Dec 2, 2016
6e1bbdd
Fix mima
tdas Dec 2, 2016
c9224ef
Fix python style
tdas Dec 2, 2016
afd5c0f
Improve docs
tdas Dec 2, 2016
7ee4cf1
Fix indent
tdas Dec 2, 2016
a2a6d90
Addressed comments
tdas Dec 3, 2016
0e3b0ac
Removed python test
tdas Dec 5, 2016
19461e1
Merge remote-tracking branch 'apache-github/master' into SPARK-18657
tdas Dec 5, 2016
4041a22
Fix test
tdas Dec 5, 2016
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Fix test
  • Loading branch information
@tdas
tdas committed Dec 5, 2016
commit 4041a2297289aa6186a07e1ea74541d625174599
11 changes: 2 additions & 9 deletions sql/core/src/test/scala/org/apache/spark/sql/streaming/StreamingQuerySuite.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -143,19 +143,12 @@ class StreamingQuerySuite extends StreamTest with BeforeAndAfter with Logging {
TestAwaitTermination(ExpectException[SparkException]),
TestAwaitTermination(ExpectException[SparkException], timeoutMs = 2000),
TestAwaitTermination(ExpectException[SparkException], timeoutMs = 10),
<<<<<<< HEAD
AssertOnQuery(
q => q.exception.get.startOffset.get.offsets ===
q.committedOffsets.toOffsetSeq(Seq(inputData), OffsetSeqMetadata()).offsets,
"incorrect start offset on exception")
=======
AssertOnQuery(q => {
q.exception.get.startOffset ===
q.committedOffsets.toOffsetSeq(Seq(inputData), "{}").toString &&
q.committedOffsets.toOffsetSeq(Seq(inputData), OffsetSeqMetadata()).toString &&
q.exception.get.endOffset ===
q.availableOffsets.toOffsetSeq(Seq(inputData), "{}").toString
q.availableOffsets.toOffsetSeq(Seq(inputData), OffsetSeqMetadata()).toString
}, "incorrect start offset or end offset on exception")
>>>>>>> apache-github/master
)
}

Expand Down