Skip to content

[SPARK-4789] [SPARK-4942] [SPARK-5031] [mllib] Standardize ML Prediction APIs #3637

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
jkbradley wants to merge 30 commits into from
Closed

[SPARK-4789] [SPARK-4942] [SPARK-5031] [mllib] Standardize ML Prediction APIs #3637

Changes from 1 commit
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
bfade12
Added lots of classes for new ML API:
jkbradley Nov 24, 2014
d35bb5d
fixed compilation issues, but have not added tests yet
jkbradley Nov 24, 2014
52f4fde
removing everything except for simple class hierarchy for classification
jkbradley Dec 1, 2014
d705e87
Added LinearRegression and Regressor back from ml-api branch
jkbradley Dec 4, 2014
601e792
Modified ParamMap to sort parameters in toString. Cleaned up classes…
jkbradley Dec 5, 2014
0617d61
Fixed bug from last commit (sorting paramMap by parameter names in to…
jkbradley Dec 5, 2014
54b7b31
Fixed issue with logreg threshold being set correctly
jkbradley Dec 5, 2014
e433872
Updated docs. Added LabeledPointSuite to spark.ml
jkbradley Dec 5, 2014
57d54ab
* Changed semantics of Predictor.train() to merge the given paramMap …
jkbradley Dec 5, 2014
58802e3
added train() to Predictor subclasses which does not take a ParamMap.
jkbradley Dec 6, 2014
adbe50a
* fixed LinearRegression train() to use embedded paramMap
jkbradley Dec 6, 2014
1680905
Added JavaLabeledPointSuite.java for spark.ml, and added constructor …
jkbradley Dec 6, 2014
8d13233
Added methods:
jkbradley Dec 8, 2014
bc654e1
Added spark.ml LinearRegressionSuite
jkbradley Dec 8, 2014
4e2f711
rat fix
jkbradley Dec 8, 2014
1c61723
* Made ProbabilisticClassificationModel into a subclass of Classifica…
jkbradley Dec 30, 2014
934f97b
Fixed bugs from previous commit.
jkbradley Dec 30, 2014
c3c8da5
small cleanup
jkbradley Dec 30, 2014
0a16da9
Fixed Linear/Logistic RegressionSuites
jkbradley Dec 31, 2014
82f340b
Fixed bug in LogisticRegression (introduced in this PR). Fixed Java …
jkbradley Dec 31, 2014
343e7bd
added blanket mima exclude for ml package
jkbradley Dec 31, 2014
f549e34
Updates based on code review. Major ones are:
jkbradley Jan 15, 2015
216d199
fixed after sql datatypes PR got merged
jkbradley Jan 15, 2015
f542997
Added MIMA excludes for VectorUDT (now public), and added DeveloperAp…
jkbradley Jan 19, 2015
9872424
fixed JavaLinearRegressionSuite.java Java sql api
jkbradley Jan 19, 2015
bcb9549
Fixed issues after rebasing from master (after move from SchemaRDD to…
jkbradley Jan 30, 2015
fc62406
fixed test suites after last commit
jkbradley Jan 30, 2015
8316d5e
fixes after rebasing on master
jkbradley Feb 5, 2015
fec348a
Added JavaDeveloperApiExample.java and fixed other issues: Made devel…
jkbradley Feb 6, 2015
405bfb8
Last edits based on code review. Small cleanups
jkbradley Feb 6, 2015
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
small cleanup
  • Loading branch information
@jkbradley
jkbradley committed Feb 5, 2015
commit c3c8da507cbfd03ed5f2fc0fff6431fbf88862ce
13 changes: 0 additions & 13 deletions mllib/src/main/scala/org/apache/spark/ml/impl/estimator/Predictor.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -80,23 +80,10 @@ abstract class Predictor[
M <: PredictionModel[FeaturesType, M]]
extends Estimator[M] with PredictorParams {

// TODO: Eliminate asInstanceOf and see if that works.
def setLabelCol(value: String): Learner = set(labelCol, value).asInstanceOf[Learner]
def setFeaturesCol(value: String): Learner = set(featuresCol, value).asInstanceOf[Learner]
def setPredictionCol(value: String): Learner = set(predictionCol, value).asInstanceOf[Learner]

/*
// This will be useful for boosting.
protected def selectLabelColumn(dataset: SchemaRDD, paramMap: ParamMap): RDD[Double] = {
import dataset.sqlContext._
val map = this.paramMap ++ paramMap
dataset.select(map(labelCol).attr).map {
case Row(label: Double) => label
case Row(label: Int) => label.toDouble
}
}
*/

/**
* :: DeveloperApi ::
*
Expand Down