Skip to content

[SPARK-23162][PySpark][ML] Add r2adj into Python API in LinearRegressionSummary #20842

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
kevinyu98 wants to merge 3 commits into from
Closed

[SPARK-23162][PySpark][ML] Add r2adj into Python API in LinearRegressionSummary #20842

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
12d18b5
add r2adj into Python API in LinearRegressionSummary
kevinyu98 Mar 16, 2018
ab0b04d
address comment
kevinyu98 Mar 24, 2018
596f58a
adjust the link name for R^2
kevinyu98 Mar 26, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions python/pyspark/ml/regression.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -347,6 +347,20 @@ def r2(self):
"""
return self._call_java("r2")

@property
@since("2.4.0")
def r2adj(self):
"""
Returns Adjusted R^2^, the adjusted coefficient of determination.
Copy link
Member

@BryanCutler BryanCutler Mar 22, 2018 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

R^2^ is a scaladoc format. How about just R^2? and could you fix in def r2 also

(R^2 would mean XOR in python, so R**2 seems the best to me unless someone knows a better format to use with Sphinx)
edit again, let's just stick with R^2, I think it's fine here and no one would think it's XOR

Copy link
Contributor Author

@kevinyu98 kevinyu98 Mar 24, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sure, I will change.


.. seealso:: `Wikipedia coefficient of determination \
Copy link
Member

@BryanCutler BryanCutler Mar 26, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like the error was because the link name is the same as above, maybe try Wikipedia coefficient of determination, Adjusted R^2

Copy link
Contributor Author

@kevinyu98 kevinyu98 Mar 26, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok, thanks.

<https://en.wikipedia.org/wiki/Coefficient_of_determination>`
Copy link
Member

@BryanCutler BryanCutler Mar 22, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

use the same link from the scaladoc https://en.wikipedia.org/wiki/Coefficient_of_determination#Adjusted_R2

It also doesn't generate properly, you need a trailing _ I believe (also need to fix def r2 if you don't mind).

it should be:

.. seealso:: `Wikipedia coefficient of determination \
<http://en.wikipedia.org/wiki/Coefficient_of_determination#Adjusted_R2>`_

If you are able to, building the docs to check these is a good idea

Copy link
Contributor Author

@kevinyu98 kevinyu98 Mar 24, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok, done.


.. note:: This ignores instance weights (setting all to 1.0) from
`LinearRegression.weightCol`. This will change in later Spark versions.
"""
return self._call_java("r2adj")

@property
@since("2.0.0")
def residuals(self):
Expand Down
1 change: 1 addition & 0 deletions python/pyspark/ml/tests.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1488,6 +1488,7 @@ def test_linear_regression_summary(self):
self.assertAlmostEqual(s.meanSquaredError, 0.0)
self.assertAlmostEqual(s.rootMeanSquaredError, 0.0)
self.assertAlmostEqual(s.r2, 1.0, 2)
self.assertAlmostEqual(s.r2adj, 1.0, 2)
self.assertTrue(isinstance(s.residuals, DataFrame))
self.assertEqual(s.numInstances, 2)
self.assertEqual(s.degreesOfFreedom, 1)
Expand Down