Skip to content

[SPARK-29235][ML][Pyspark]Support avgMetrics in read/write of CrossValidatorModel #26038

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
shahidki31 wants to merge 8 commits into from
Closed

[SPARK-29235][ML][Pyspark]Support avgMetrics in read/write of CrossValidatorModel #26038

Changes from 1 commit
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
1c594da
Support avgMetrics
shahidki31 Oct 6, 2019
13e3a59
style
shahidki31 Oct 6, 2019
6068f66
update
shahidki31 Oct 17, 2019
b0f1975
style
shahidki31 Oct 17, 2019
5a79a8a
conflict resolve
shahidki31 Oct 17, 2019
5367e54
Merge branch 'master' of https://github.com/apache/spark into avgMetrics
shahidki31 Oct 17, 2019
2755376
minor update
shahidki31 Oct 18, 2019
00c4258
address comment
shahidki31 Oct 18, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
minor update
  • Loading branch information
@shahidki31
shahidki31 committed Oct 18, 2019
commit 2755376191c5b7e479001b36f6041f88d407d85c
4 changes: 2 additions & 2 deletions python/pyspark/ml/tuning.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -217,6 +217,7 @@ class CrossValidator(Estimator, _CrossValidatorParams, HasParallelism, HasCollec
>>> from pyspark.ml.evaluation import BinaryClassificationEvaluator
>>> from pyspark.ml.linalg import Vectors
>>> from pyspark.ml.tuning import CrossValidatorModel
>>> import tempfile
>>> dataset = spark.createDataFrame(
... [(Vectors.dense([0.0]), 0.0),
... (Vectors.dense([0.4]), 1.0),
Expand All @@ -234,7 +235,6 @@ class CrossValidator(Estimator, _CrossValidatorParams, HasParallelism, HasCollec
3
>>> cvModel.avgMetrics[0]
0.5
>>> import tempfile
>>> path = tempfile.mkdtemp()
>>> model_path = path + "/model"
>>> cvModel.write().save(model_path)
Expand Down Expand Up @@ -560,6 +560,7 @@ class TrainValidationSplit(Estimator, _TrainValidationSplitParams, HasParallelis
>>> from pyspark.ml.evaluation import BinaryClassificationEvaluator
>>> from pyspark.ml.linalg import Vectors
>>> from pyspark.ml.tuning import TrainValidationSplitModel
>>> import tempfile
>>> dataset = spark.createDataFrame(
... [(Vectors.dense([0.0]), 0.0),
... (Vectors.dense([0.4]), 1.0),
Expand All @@ -577,7 +578,6 @@ class TrainValidationSplit(Estimator, _TrainValidationSplitParams, HasParallelis
0.75
>>> tvsModel.validationMetrics
[0.5, ...
>>> import tempfile
>>> path = tempfile.mkdtemp()
>>> model_path = path + "/model"
>>> tvsModel.write().save(model_path)
Expand Down