Skip to content

[SPARK-29377][PYTHON][ML] Parity between Scala ML tuning and Python ML tuning #26057

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
huaxingao wants to merge 2 commits into from
Closed

[SPARK-29377][PYTHON][ML] Parity between Scala ML tuning and Python ML tuning #26057

Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
8a5615b
[SPARK-29377][PYTHON][ML] Parity between scala ml tuning and python m…
huaxingao Oct 8, 2019
2a23e08
add version info in getters/setters
huaxingao Oct 13, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
add version info in getters/setters
  • Loading branch information
@huaxingao
huaxingao committed Oct 13, 2019
commit 2a23e08654d9c8b36f5ee38d58186067e3cf64cd
15 changes: 15 additions & 0 deletions python/pyspark/ml/tuning.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -133,18 +133,21 @@ class _ValidatorParams(HasSeed):
Params._dummy(), "evaluator",
"evaluator used to select hyper-parameters that maximize the validator metric")

@since("2.0.0")
def getEstimator(self):
"""
Gets the value of estimator or its default value.
"""
return self.getOrDefault(self.estimator)

@since("2.0.0")
def getEstimatorParamMaps(self):
"""
Gets the value of estimatorParamMaps or its default value.
"""
return self.getOrDefault(self.estimatorParamMaps)

@since("2.0.0")
def getEvaluator(self):
"""
Gets the value of evaluator or its default value.
Expand Down Expand Up @@ -260,18 +263,21 @@ def setParams(self, estimator=None, estimatorParamMaps=None, evaluator=None, num
kwargs = self._input_kwargs
return self._set(**kwargs)

@since("2.0.0")
def setEstimator(self, value):
"""
Sets the value of :py:attr:`estimator`.
"""
return self._set(estimator=value)

@since("2.0.0")
def setEstimatorParamMaps(self, value):
"""
Sets the value of :py:attr:`estimatorParamMaps`.
"""
return self._set(estimatorParamMaps=value)

@since("2.0.0")
def setEvaluator(self, value):
Copy link
Member

@viirya viirya Oct 12, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to add since for setters? Some like setNumFolds have it, but some do not.

Copy link
Contributor Author

@huaxingao huaxingao Oct 12, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the original file has @SInCE, I have it too. Some of the getters/setters don't have @SInCE in the original file, so I don't have i too.
I will check the history to find out which release starts to have these getters/setters, and add @SInCE for each of the getters/setters.

Copy link
Contributor Author

@huaxingao huaxingao Oct 12, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I checked the history, the @since was removed in #12020. I guess the reason is that in CrossValidator, the getters/getters have @since 1.4, but in TrainValidationSplit, the getters/getters have @since 2.0. When putting the common getters/setters in ValidatorParams, neither @since 1.4 nor @since 2.0 is quite right, so @since was removed. I am facing the same problem. Shall I have @since 2.0? or just not adding back the @since?

Copy link
Member

@srowen srowen Oct 13, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@since 2.0 is at least accurate, as both have existed since that version. One existed before, sure, but this is maybe not a big deal, as it won't matter much at all to know it was in 1.x.

"""
Sets the value of :py:attr:`evaluator`.
Expand Down Expand Up @@ -417,18 +423,21 @@ def __init__(self, bestModel, avgMetrics=[], subModels=None):
#: sub model list from cross validation
self.subModels = subModels

@since("2.0.0")
def setEstimator(self, value):
"""
Sets the value of :py:attr:`estimator`.
"""
return self._set(estimator=value)

@since("2.0.0")
def setEstimatorParamMaps(self, value):
"""
Sets the value of :py:attr:`estimatorParamMaps`.
"""
return self._set(estimatorParamMaps=value)

@since("2.0.0")
def setEvaluator(self, value):
"""
Sets the value of :py:attr:`evaluator`.
Expand Down Expand Up @@ -587,18 +596,21 @@ def setParams(self, estimator=None, estimatorParamMaps=None, evaluator=None, tra
kwargs = self._input_kwargs
return self._set(**kwargs)

@since("2.0.0")
def setEstimator(self, value):
"""
Sets the value of :py:attr:`estimator`.
"""
return self._set(estimator=value)

@since("2.0.0")
def setEstimatorParamMaps(self, value):
"""
Sets the value of :py:attr:`estimatorParamMaps`.
"""
return self._set(estimatorParamMaps=value)

@since("2.0.0")
def setEvaluator(self, value):
"""
Sets the value of :py:attr:`evaluator`.
Expand Down Expand Up @@ -734,18 +746,21 @@ def __init__(self, bestModel, validationMetrics=[], subModels=None):
#: sub models from train validation split
self.subModels = subModels

@since("2.0.0")
def setEstimator(self, value):
"""
Sets the value of :py:attr:`estimator`.
"""
return self._set(estimator=value)

@since("2.0.0")
def setEstimatorParamMaps(self, value):
"""
Sets the value of :py:attr:`estimatorParamMaps`.
"""
return self._set(estimatorParamMaps=value)

@since("2.0.0")
def setEvaluator(self, value):
"""
Sets the value of :py:attr:`evaluator`.
Expand Down