-
Notifications
You must be signed in to change notification settings - Fork 29k
[SPARK-17748][ML] Minor cleanups to one-pass linear regression with elastic net #15779
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 1 commit
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
- Loading branch information
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -160,14 +160,13 @@ class LinearRegression @Since("1.3.0") (@Since("1.3.0") override val uid: String | |
| /** | ||
| * Set the solver algorithm used for optimization. | ||
| * In case of linear regression, this can be "l-bfgs", "normal" and "auto". | ||
| * "l-bfgs" denotes Limited-memory BFGS which is a limited-memory quasi-Newton | ||
| * optimization method. "normal" denotes using Normal Equation as an analytical | ||
| * solution to the linear regression problem. | ||
| * The default value is "auto" which means that the solver algorithm is | ||
| * selected automatically. | ||
| * | ||
| * The Normal Equation solver is limited to a few thousand features; when needed, | ||
| * [[LinearRegression]] will automatically fall back to iterative optimization methods. | ||
| * - "l-bfgs" denotes Limited-memory BFGS which is a limited-memory quasi-Newton | ||
| * optimization method. | ||
| * - "normal" denotes using Normal Equation as an analytical solution to the linear regression | ||
| * problem. This solver is limited to [[LinearRegression.MAX_FEATURES_FOR_NORMAL_SOLVER]]. | ||
| * - "auto" (default) means that the solver algorithm is selected automatically. | ||
| * The Normal Equations solver will be used when possible, but this will automatically fall | ||
| * back to iterative optimization methods when needed. | ||
| * | ||
| * @group setParam | ||
| */ | ||
|
|
@@ -407,6 +406,13 @@ object LinearRegression extends DefaultParamsReadable[LinearRegression] { | |
|
|
||
| @Since("1.6.0") | ||
| override def load(path: String): LinearRegression = super.load(path) | ||
|
|
||
| /** | ||
| * When using [[LinearRegression.solver]] == "normal", the solver must limit the number of | ||
| * features to at most this number. | ||
| */ | ||
| @Since("2.1.0") | ||
| val MAX_FEATURES_FOR_NORMAL_SOLVER: Int = WeightedLeastSquares.MAX_NUM_FEATURES | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I prefer the current name.
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yep, I left out "NUM_" for conciseness. |
||
| } | ||
|
|
||
| /** | ||
|
|
||
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
minor suggestion: perhaps an explanation as to why might be useful. "must limit the number of features to at most this number, because the entire covariance matrix
X^T^Xwill be collected on the driver. This limit helps prevent memory overflow errors."There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Will do