Skip to content

[SPARK-18324][ML][DOC] Update ML programming and migration guide for 2.1 release #16076

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
yanboliang wants to merge 3 commits into from
+163 −134
Closed

[SPARK-18324][ML][DOC] Update ML programming and migration guide for 2.1 release #16076

Changes from 1 commit
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
f3f1fc3
Programming guide update and migration guide
yanboliang Nov 30, 2016
e784901
Address comments.
yanboliang Dec 1, 2016
f583653
Remove SPARK-18481 from breaking changes.
yanboliang Dec 2, 2016
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Address comments.
  • Loading branch information
@yanboliang
yanboliang committed Dec 1, 2016
commit e784901f1e2b1cd10c15537efa28077f8e67a768
14 changes: 8 additions & 6 deletions docs/ml-guide.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -64,33 +64,35 @@ and the migration guide below will explain all changes between releases.

### Breaking changes

* [SPARK-18481](https://issues.apache.org/jira/browse/SPARK-18481):
* `RandomForestClassificationModel.getNumTrees` and `RandomForestRegressionModel.getNumTrees` were made final.
* `RandomForestClassificationModel.setFeatureSubsetStrategy` and `RandomForestRegressionModel.setFeatureSubsetStrategy` return the concrete class type,
rather than an arbitrary trait. This only affected Java compatibility, not Scala.

**Deprecated methods removed**

* `setLabelCol` in `feature.ChiSqSelectorModel`
* `numTrees` in `classification.RandomForestClassificationModel` (This now refers to the Param called `numTrees`)
* `numTrees` in `regression.RandomForestRegressionModel` (This now refers to the Param called `numTrees`)
* `model` in `regression.LinearRegressionSummary`
* `validateParams` in `PipelineStage`
Copy link
Member

@jkbradley jkbradley Nov 30, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also: validateParams in Evaluator

* `validateParams` in `Evaluator`

### Deprecations and changes of behavior

**Deprecations**

* [SPARK-18592](https://issues.apache.org/jira/browse/SPARK-18592):
Deprecate all setter methods for `DecisionTreeClassificationModel`, `GBTClassificationModel`, `RandomForestClassificationModel`, `DecisionTreeRegressionModel`, `GBTRegressionModel` and `RandomForestRegressionModel`
Deprecate all Param setter methods except for input/output column Params for `DecisionTreeClassificationModel`, `GBTClassificationModel`, `RandomForestClassificationModel`, `DecisionTreeRegressionModel`, `GBTRegressionModel` and `RandomForestRegressionModel`

**Changes of behavior**

* [SPARK-17870](https://issues.apache.org/jira/browse/SPARK-17870):
Fix a bug of `ChiSqSelector` which will likely change its result.
Fix a bug of `ChiSqSelector` which will likely change its result. Now `ChiSquareSelector` use pValue rather than raw statistic to select a fixed number of top features.
* [SPARK-3261](https://issues.apache.org/jira/browse/SPARK-3261):
`KMeans` returns potentially fewer than k cluster centers in cases where k distinct centroids aren't available or aren't selected.
* [SPARK-17389](https://issues.apache.org/jira/browse/SPARK-17389):
`KMeans` reduces the default number of steps from 5 to 2 for the k-means|| initialization mode.
* [SPARK-18481](https://issues.apache.org/jira/browse/SPARK-18481):
* `RandomForestClassificationModel.getNumTrees` and `RandomForestRegressionModel.getNumTrees` were made final.
* `RandomForestClassificationModel.setFeatureSubsetStrategy` and `RandomForestRegressionModel.setFeatureSubsetStrategy` return the concrete class type,
rather than an arbitrary trait. This only affected Java compatibility, not Scala.

## Previous Spark versions

Expand Down