Skip to content

[SPARK-30119][WebUI]Support Pagination for Batch Tables in Streaming Tab #26756

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
iRakson wants to merge 5 commits into from
Closed

[SPARK-30119][WebUI]Support Pagination for Batch Tables in Streaming Tab #26756

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
69b8a40
[SPARK-30119]Support Pagination for Completed Batch Table in Streamin…
iRakson Dec 4, 2019
c7cfde2
[SPARK-30119] Streaming Pagination: Fix Review Comments
iRakson Dec 5, 2019
cdea7c3
[SPARK-30119] Streaming Pagination: Fix Review Comments
iRakson Dec 6, 2019
84a5e77
[SPARK-30234]Fix Review Comments
iRakson Dec 14, 2019
7ec80b9
[SPARK-30119]Fix
iRakson Dec 15, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
[SPARK-30119]Fix
  • Loading branch information
@iRakson
iRakson committed Dec 15, 2019
commit 7ec80b920014ac326fd5b23e6699eca040111a2f
3 changes: 0 additions & 3 deletions streaming/src/main/scala/org/apache/spark/streaming/ui/AllBatchesTable.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,9 +39,7 @@ private[ui] class StreamingBatchPagedTable(
desc: Boolean) extends PagedTable[BatchUIData] {

override val dataSource = new StreamingBatchTableDataSource(batchData, pageSize, sortColumn, desc)

private val parameterPath = s"$basePath/$subPath/?${parameterOtherTable.mkString("&")}"

private val firstFailureReason = getFirstFailureReason(batchData)

override def tableId: String = streamingBatchTag
Expand Down Expand Up @@ -200,7 +198,6 @@ private[ui] class StreamingBatchTableDataSource(
desc: Boolean) extends PagedDataSource[BatchUIData](pageSize) {

private val data = info.sorted(ordering(sortColumn, desc))

private var _slicedStartTime: Set[Long] = null

override def dataSize: Int = data.size
Expand Down
2 changes: 0 additions & 2 deletions streaming/src/main/scala/org/apache/spark/streaming/ui/StreamingPage.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -499,7 +499,6 @@ private[ui] class StreamingPage(parent: StreamingTab)
val parameterStreamingBatchSortColumn = request.getParameter(streamingBatchTag + ".sort")
val parameterStreamingBatchSortDesc = request.getParameter(streamingBatchTag + ".desc")
val parameterStreamingBatchPageSize = request.getParameter(streamingBatchTag + ".pageSize")

val streamingBatchPage = Option(parameterStreamingBatchPage).map(_.toInt).getOrElse(1)
val streamingBatchSortColumn = Option(parameterStreamingBatchSortColumn).map { sortColumn =>
SparkUIUtils.decodeURLParameter(sortColumn)
Expand Down Expand Up @@ -579,7 +578,6 @@ private[ui] class StreamingPage(parent: StreamingTab)
</div>
</div>
}

activeBatchesContent ++ completedBatchesContent
}
}
Expand Down