Skip to content

[SPARK-22897][CORE]: Expose stageAttemptId in TaskContext #20082

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
advancedxy wants to merge 6 commits into from
Closed

[SPARK-22897][CORE]: Expose stageAttemptId in TaskContext #20082

Changes from 1 commit
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
5753ee0
[SPARK-22897][CORE]: Expose stageAttemptId in TaskContext
advancedxy Dec 26, 2017
f02bc1e
Minor fixes such as:
advancedxy Dec 26, 2017
59e4a9c
Update MimaExcludes to move stageAttemptId filter to the beginning
advancedxy Dec 27, 2017
291bbbc
Add comment for usage of FetchFailedException
advancedxy Dec 27, 2017
72a3abf
Add override for stageId, stageAttemptId and partitionId.
advancedxy Dec 28, 2017
9266cd8
Rename stageAttemptId to stageAttemptNumber
advancedxy Dec 29, 2017
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Update MimaExcludes to move stageAttemptId filter to the beginning
  • Loading branch information
@advancedxy
advancedxy committed Dec 27, 2017
commit 59e4a9c70c037729f3eb60b47b2e625208687385
8 changes: 4 additions & 4 deletions project/MimaExcludes.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,9 @@ object MimaExcludes {

// Exclude rules for 2.3.x
lazy val v23excludes = v22excludes ++ Seq(
// [SPARK-22897] Expose stageAttemptId in TaskContext
ProblemFilters.exclude[ReversedMissingMethodProblem]("org.apache.spark.TaskContext.stageAttemptId"),

// SPARK-22789: Map-only continuous processing execution
ProblemFilters.exclude[IncompatibleResultTypeProblem]("org.apache.spark.sql.streaming.StreamingQueryManager.startQuery$default$8"),
ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.sql.streaming.StreamingQueryManager.startQuery$default$6"),
Expand Down Expand Up @@ -95,10 +98,7 @@ object MimaExcludes {

// [SPARK-21087] CrossValidator, TrainValidationSplit expose sub models after fitting: Scala
ProblemFilters.exclude[FinalClassProblem]("org.apache.spark.ml.tuning.CrossValidatorModel$CrossValidatorModelWriter"),
ProblemFilters.exclude[FinalClassProblem]("org.apache.spark.ml.tuning.TrainValidationSplitModel$TrainValidationSplitModelWriter"),

// [SPARK-22897] Expose stageAttemptId in TaskContext
ProblemFilters.exclude[ReversedMissingMethodProblem]("org.apache.spark.TaskContext.stageAttemptId")
ProblemFilters.exclude[FinalClassProblem]("org.apache.spark.ml.tuning.TrainValidationSplitModel$TrainValidationSplitModelWriter")
)

// Exclude rules for 2.2.x
Expand Down