Skip to content

[SPARK-2852][MLLIB] API consistency for mllib.feature #1807

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
mengxr wants to merge 2 commits into from
Closed

[SPARK-2852][MLLIB] API consistency for mllib.feature #1807

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
883e122
add @Experimental to Word2VecModel
mengxr Aug 6, 2014
773c1a9
change default numFeatures to 2^20 in HashingTF
mengxr Aug 6, 2014
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
change default numFeatures to 2^20 in HashingTF 
change annotation from DeveloperApi to Experimental in Normalizer and StandardScaler
  • Loading branch information
@mengxr
mengxr committed Aug 6, 2014
commit 773c1a9a2d231dbb6565338569cf49ad95a6777e
4 changes: 2 additions & 2 deletions mllib/src/main/scala/org/apache/spark/mllib/feature/HashingTF.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,12 +32,12 @@ import org.apache.spark.util.Utils
* :: Experimental ::
* Maps a sequence of terms to their term frequencies using the hashing trick.
*
* @param numFeatures number of features (default: 1000000)
* @param numFeatures number of features (default: 2^20^)
*/
@Experimental
class HashingTF(val numFeatures: Int) extends Serializable {

def this() = this(1000000)
def this() = this(1 << 20)

/**
* Returns the index of the input term.
Expand Down
6 changes: 3 additions & 3 deletions mllib/src/main/scala/org/apache/spark/mllib/feature/Normalizer.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,11 +19,11 @@ package org.apache.spark.mllib.feature

import breeze.linalg.{DenseVector => BDV, SparseVector => BSV}

import org.apache.spark.annotation.DeveloperApi
import org.apache.spark.annotation.Experimental
import org.apache.spark.mllib.linalg.{Vector, Vectors}

/**
* :: DeveloperApi ::
* :: Experimental ::
* Normalizes samples individually to unit L^p^ norm
*
* For any 1 <= p < Double.PositiveInfinity, normalizes samples using
Expand All @@ -33,7 +33,7 @@ import org.apache.spark.mllib.linalg.{Vector, Vectors}
*
* @param p Normalization in L^p^ space, p = 2 by default.
*/
@DeveloperApi
@Experimental
class Normalizer(p: Double) extends VectorTransformer {

def this() = this(2)
Expand Down
6 changes: 3 additions & 3 deletions mllib/src/main/scala/org/apache/spark/mllib/feature/StandardScaler.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,22 +19,22 @@ package org.apache.spark.mllib.feature

import breeze.linalg.{DenseVector => BDV, SparseVector => BSV, Vector => BV}

import org.apache.spark.annotation.DeveloperApi
import org.apache.spark.annotation.Experimental
import org.apache.spark.mllib.linalg.{Vector, Vectors}
import org.apache.spark.mllib.rdd.RDDFunctions._
import org.apache.spark.mllib.stat.MultivariateOnlineSummarizer
import org.apache.spark.rdd.RDD

/**
* :: DeveloperApi ::
* :: Experimental ::
* Standardizes features by removing the mean and scaling to unit variance using column summary
* statistics on the samples in the training set.
*
* @param withMean False by default. Centers the data with mean before scaling. It will build a
* dense output, so this does not work on sparse input and will raise an exception.
* @param withStd True by default. Scales the data to unit standard deviation.
*/
@DeveloperApi
@Experimental
class StandardScaler(withMean: Boolean, withStd: Boolean) extends VectorTransformer {

def this() = this(false, true)
Expand Down