Skip to content

[SPARK-31764][CORE] JsonProtocol doesn't write RDDInfo#isBarrier #28583

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
sarutak wants to merge 3 commits into from
Closed

[SPARK-31764][CORE] JsonProtocol doesn't write RDDInfo#isBarrier #28583

Changes from 1 commit
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
0816958
Write barrier information to event log.
sarutak May 19, 2020
5851842
Added a testcase.
sarutak May 19, 2020
53ebb55
Modified util.JsonProtocolSuite to comply with this change.
sarutak May 21, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Next Next commit
Write barrier information to event log.
  • Loading branch information
@sarutak
sarutak committed May 19, 2020
commit 081695819b78c74d2fbce4dcd8a18f14dbb2dfd3
1 change: 1 addition & 0 deletions core/src/main/scala/org/apache/spark/util/JsonProtocol.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -475,6 +475,7 @@ private[spark] object JsonProtocol {
("Callsite" -> rddInfo.callSite) ~
("Parent IDs" -> parentIds) ~
("Storage Level" -> storageLevel) ~
("Barrier" -> rddInfo.isBarrier) ~
("Number of Partitions" -> rddInfo.numPartitions) ~
("Number of Cached Partitions" -> rddInfo.numCachedPartitions) ~
("Memory Size" -> rddInfo.memSize) ~
Expand Down