Skip to content

SKIPME merged Apache branch-1.6 #132

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
markhamstra merged 22 commits into from
Dec 16, 2015
Merged

SKIPME merged Apache branch-1.6 #132

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
23c8846
[STREAMING][MINOR] Fix typo in function name of StateImpl
jerryshao Dec 15, 2015
80d2617
Update branch-1.6 for 1.6.0 release
marmbrus Dec 15, 2015
00a39d9
Preparing Spark release v1.6.0-rc3
pwendell Dec 15, 2015
08aa3b4
Preparing development version 1.6.0-SNAPSHOT
pwendell Dec 15, 2015
9e4ac56
[SPARK-12056][CORE] Part 2 Create a TaskAttemptContext only after cal…
tedyu Dec 16, 2015
2c324d3
[SPARK-12351][MESOS] Add documentation about submitting Spark with me…
tnachen Dec 16, 2015
8e9a600
[SPARK-9886][CORE] Fix to use ShutdownHookManager in
naveenminchu Dec 16, 2015
93095eb
[SPARK-12062][CORE] Change Master to asyc rebuild UI when application…
BryanCutler Dec 16, 2015
fb08f7b
[SPARK-10477][SQL] using DSL in ColumnPruningSuite to improve readabi…
cloud-fan Dec 16, 2015
135a5ee
removed some maven-jar-plugin
markhamstra Dec 16, 2015
9a6494a
Merge branch 'branch-1.6' of github.com:apache/spark into csd-1.6
markhamstra Dec 16, 2015
a2d584e
[SPARK-12324][MLLIB][DOC] Fixes the sidebar in the ML documentation
thunterdb Dec 16, 2015
ac0e2ea
[SPARK-12310][SPARKR] Add write.json and write.parquet for SparkR
yanboliang Dec 16, 2015
16edd93
[SPARK-12215][ML][DOC] User guide section for KMeans in spark.ml
yu-iskw Dec 16, 2015
f815127
[SPARK-12318][SPARKR] Save mode in SparkR should be error by default
zjffdu Dec 16, 2015
e5b8571
[SPARK-12345][MESOS] Filter SPARK_HOME when submitting Spark jobs wit…
tnachen Dec 16, 2015
e1adf6d
[SPARK-6518][MLLIB][EXAMPLE][DOC] Add example code and user guide for…
yu-iskw Dec 16, 2015
168c89e
Preparing Spark release v1.6.0-rc3
pwendell Dec 16, 2015
aee88eb
Preparing development version 1.6.0-SNAPSHOT
pwendell Dec 16, 2015
dffa610
[SPARK-11608][MLLIB][DOC] Added migration guide for MLlib 1.6
jkbradley Dec 16, 2015
04e868b
[SPARK-12364][ML][SPARKR] Add ML example for SparkR
yanboliang Dec 16, 2015
d020431
Merge branch 'branch-1.6' of github.com:apache/spark into csd-1.6
markhamstra Dec 16, 2015
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Next Next commit
[STREAMING][MINOR] Fix typo in function name of StateImpl 
cc\ tdas zsxwing , please review. Thanks a lot.

Author: jerryshao <[email protected]>

Closes apache#10305 from jerryshao/fix-typo-state-impl.

(cherry picked from commit bc1ff9f)
Signed-off-by: Shixiong Zhu <[email protected]>
  • Loading branch information
@jerryshao @zsxwing
jerryshao authored and zsxwing committed Dec 15, 2015
commit 23c8846050b307fdfe2307f7e7ca9d0f69f969a9
2 changes: 1 addition & 1 deletion streaming/src/main/scala/org/apache/spark/streaming/State.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -206,7 +206,7 @@ private[streaming] class StateImpl[S] extends State[S] {
* Update the internal data and flags in `this` to the given state that is going to be timed out.
* This method allows `this` object to be reused across many state records.
*/
def wrapTiminoutState(newState: S): Unit = {
def wrapTimingOutState(newState: S): Unit = {
this.state = newState
defined = true
timingOut = true
Expand Down
2 changes: 1 addition & 1 deletion streaming/src/main/scala/org/apache/spark/streaming/rdd/MapWithStateRDD.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@ private[streaming] object MapWithStateRDDRecord {
// data returned
if (removeTimedoutData && timeoutThresholdTime.isDefined) {
newStateMap.getByTime(timeoutThresholdTime.get).foreach { case (key, state, _) =>
wrappedState.wrapTiminoutState(state)
wrappedState.wrapTimingOutState(state)
val returned = mappingFunction(batchTime, key, None, wrappedState)
mappedData ++= returned
newStateMap.remove(key)
Expand Down
2 changes: 1 addition & 1 deletion streaming/src/test/scala/org/apache/spark/streaming/MapWithStateSuite.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -125,7 +125,7 @@ class MapWithStateSuite extends SparkFunSuite
state.remove()
testState(None, shouldBeRemoved = true)

state.wrapTiminoutState(3)
state.wrapTimingOutState(3)
testState(Some(3), shouldBeTimingOut = true)
}

Expand Down