Skip to content

[SPARK-9835] [ML] Implement IterativelyReweightedLeastSquares solver #10639

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
yanboliang wants to merge 12 commits into from
Closed

[SPARK-9835] [ML] Implement IterativelyReweightedLeastSquares solver #10639

Changes from 1 commit
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
b7d24c5
Initial draft of IterativelyReweightedLeastSquares
yanboliang Jan 7, 2016
ca4f16d
fix typos
yanboliang Jan 8, 2016
abed2f3
Make Link as object
yanboliang Jan 8, 2016
f1485ca
Add wrapper function: predict & fitted
yanboliang Jan 8, 2016
d7653d6
Rename z to adjusted
yanboliang Jan 11, 2016
65133ad
add Binomial and Poisson object
yanboliang Jan 11, 2016
446b1cb
update calculation of Family.weights.
yanboliang Jan 12, 2016
6d108cd
IRLS refactor
yanboliang Jan 21, 2016
8ecb763
Update doc
yanboliang Jan 21, 2016
2191d2a
Fix typos
yanboliang Jan 21, 2016
6074fd6
clean IRLS tests
yanboliang Jan 25, 2016
cb2057e
make the convergence check more efficient
yanboliang Jan 28, 2016
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Make Link as object
  • Loading branch information
@yanboliang
yanboliang committed Jan 11, 2016
commit abed2f36ea3e0c9326f820a44035952f2035b1fa
10 changes: 5 additions & 5 deletions mllib/src/main/scala/org/apache/spark/ml/optim/GLMFamilies.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ private[ml] abstract class Family(val link: Link) extends Serializable {
* The default link for the Binomial family is the logit link.
* @param link a link function instance
*/
private[ml] class Binomial(link: Link = new Logit) extends Family(link) {
private[ml] class Binomial(link: Link = Logit) extends Family(link) {

override def startingMu(y: Double, yMean: Double): Double = (y + 0.5) / 2.0

Expand All @@ -74,7 +74,7 @@ private[ml] class Binomial(link: Link = new Logit) extends Family(link) {
* The default link for the Poisson family is the log link.
* @param link a link function instance
*/
private[ml] class Poisson(link: Link = new Log) extends Family(link) {
private[ml] class Poisson(link: Link = Log) extends Family(link) {

override def deviance(y: RDD[Double], mu: RDD[Double]): Double = {
mu.zip(y).map { case (mu, y) =>
Expand All @@ -92,7 +92,7 @@ private[ml] class Poisson(link: Link = new Log) extends Family(link) {
/**
* A description of the link function to be used in the model.
*/
private[ml] abstract class Link extends Serializable {
private[ml] trait Link extends Serializable {

/** The link function. */
def link(mu: Double): Double
Expand All @@ -104,7 +104,7 @@ private[ml] abstract class Link extends Serializable {
def unlink(eta: Double): Double
}

private[ml] class Logit extends Link {
private[ml] object Logit extends Link {

override def link(mu: Double): Double = math.log(mu / (1.0 - mu))

Expand All @@ -113,7 +113,7 @@ private[ml] class Logit extends Link {
override def unlink(eta: Double): Double = 1.0 / (1.0 + math.exp(-1.0 * eta))
}

private[ml] class Log extends Link {
private[ml] object Log extends Link {

override def link(mu: Double): Double = math.log(mu)

Expand Down