Skip to content

[SPARK-23975][ML]Allow Clustering to take Arrays of Double as input features #21081

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
lu-wang-dl wants to merge 9 commits into from
Closed

[SPARK-23975][ML]Allow Clustering to take Arrays of Double as input features #21081

Changes from 1 commit
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
ed890d3
add Array input support for KMeans
lu-wang-dl Apr 16, 2018
badb0cc
remove redundent code
lu-wang-dl Apr 16, 2018
6d222a3
make sure the code works for Float type and add the unit test
lu-wang-dl Apr 16, 2018
009b918
consolidating featuretovector
lu-wang-dl Apr 19, 2018
cd988c7
change featureToVector to KMeanParams and add the scala docs
lu-wang-dl Apr 20, 2018
3ffb322
change featureToVector and validateSchema to private functions
lu-wang-dl Apr 23, 2018
fee36ad
Merge branch 'master' of https://github.com/apache/spark into SPARK-2…
lu-wang-dl Apr 23, 2018
3e012fb
move featureToVector to util, so that other methods could use it to a…
lu-wang-dl Apr 23, 2018
c4e1a51
fix the comments and correct code style
lu-wang-dl Apr 23, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
change featureToVector and validateSchema to private functions
  • Loading branch information
@lu-wang-dl
lu-wang-dl committed Apr 23, 2018
commit 3ffb32291503a628c63a7014d27f2313f04c5497
4 changes: 2 additions & 2 deletions mllib/src/main/scala/org/apache/spark/ml/clustering/KMeans.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -88,7 +88,7 @@ private[clustering] trait KMeansParams extends Params with HasMaxIter with HasFe
* Validates the input schema.
* @param schema input schema
*/
protected def validateSchema(schema: StructType): Unit = {
private[clustering] def validateSchema(schema: StructType): Unit = {
val typeCandidates = List( new VectorUDT,
new ArrayType(DoubleType, false),
new ArrayType(FloatType, false))
Expand All @@ -111,7 +111,7 @@ private[clustering] trait KMeansParams extends Params with HasMaxIter with HasFe
* @return Vector feature column
*/
@Since("2.4.0")
protected def featureToVector(dataset: Dataset[_], colName: String): Column = {
private[clustering] def featureToVector(dataset: Dataset[_], colName: String): Column = {
val featuresDataType = dataset.schema(colName).dataType
featuresDataType match {
case _: VectorUDT => col(colName)
Expand Down