Skip to content

[SPARK-8018][MLlib]KMeans should accept initial cluster centers as param #6737

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
FlytxtRnD wants to merge 16 commits into from
Closed

[SPARK-8018][MLlib]KMeans should accept initial cluster centers as param #6737

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
6959861
Accept initial cluster centers in KMeans
FlytxtRnD Jun 10, 2015
e9c35d7
Remove getInitialModel and match cluster count criteria
FlytxtRnD Jun 11, 2015
16f1b53
Merge branch 'Kmeans-8018', remote-tracking branch 'upstream/master' …
FlytxtRnD Jun 15, 2015
cd5dc5c
Merge remote-tracking branch 'upstream/master' into Kmeans-8018
FlytxtRnD Jun 19, 2015
3f5fc8e
test case modified and one runs condition added
FlytxtRnD Jun 19, 2015
582e6d9
Merge remote-tracking branch 'upstream/master' into Kmeans-8018
FlytxtRnD Jun 22, 2015
60c8ce2
ignore runs parameter and initialModel test suite changed
FlytxtRnD Jun 22, 2015
714acb5
added numRuns
FlytxtRnD Jun 25, 2015
242ead1
corrected == to === in assert
FlytxtRnD Jun 29, 2015
e721dfe
Merge remote-tracking branch 'upstream/master' into Kmeans-8018
FlytxtRnD Jun 29, 2015
07f8554
remove setRuns from setIntialModel
FlytxtRnD Jun 30, 2015
d12336e
numRuns variable modifications
FlytxtRnD Jul 3, 2015
06d13ef
numRuns corrected
FlytxtRnD Jul 6, 2015
c446c58
documentation and numRuns warning change
FlytxtRnD Jul 8, 2015
ef95ee2
style correction
FlytxtRnD Jul 14, 2015
94b56df
style correction
FlytxtRnD Jul 14, 2015
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
documentation and numRuns warning change
  • Loading branch information
@FlytxtRnD
FlytxtRnD committed Jul 8, 2015
commit c446c58da68b259f34485bb656c303bdaf6a458d
1 change: 1 addition & 0 deletions docs/mllib-clustering.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@ guaranteed to find a globally optimal solution, and when run multiple times on
a given dataset, the algorithm returns the best clustering result).
* *initializationSteps* determines the number of steps in the k-means\|\| algorithm.
* *epsilon* determines the distance threshold within which we consider k-means to have converged.
* *initialModel* is an optional set of cluster centers used for initialization. If this parameter is supplied, only one run is performed.

**Examples**

Expand Down
9 changes: 6 additions & 3 deletions mllib/src/main/scala/org/apache/spark/mllib/clustering/KMeans.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -209,12 +209,15 @@ class KMeans private (
val initStartTime = System.nanoTime()

// Only one run is allowed when initialModel is given
val numRuns = if (initialModel.nonEmpty) 1 else runs
logWarning("Ignoring runs; one run is allowed when initialModel is given.")
val numRuns = if (initialModel.nonEmpty){
if (runs >1 ) logWarning("Ignoring runs; one run is allowed when initialModel is given.")
Copy link
Member

@jkbradley jkbradley Jul 9, 2015

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please be careful about Scala style. Look at [https://cwiki.apache.org/confluence/display/SPARK/Spark+Code+Style+Guide] or other parts of the codebase for examples. We try hard to keep a consistent style. Here:

val numRuns = if (initialModel.nonEmpty) {
  if (runs > 1) logWarning("Ignoring runs; one run is allowed when initialModel is given.")
  1
} else {
  runs
}

Copy link
Contributor Author

@FlytxtRnD FlytxtRnD Jul 9, 2015

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jkbradley the if statement is given 2 space indentation..tat's correct rgt?

Copy link
Member

@jkbradley jkbradley Jul 9, 2015

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's correct.

Copy link
Member

@jkbradley jkbradley Jul 13, 2015

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please replace your current code with the snippet I wrote above (defining numRuns). The current code does not follow the style guide.

1
} else runs


val centers = initialModel match {
case Some(kMeansCenters) => {
Array(kMeansCenters.clusterCenters.map(s => new VectorWithNorm(s, Vectors.norm(s, 2.0))))
Array(kMeansCenters.clusterCenters.map(s => new VectorWithNorm(s)))
}
case None => {
if (initializationMode == KMeans.RANDOM) {
Expand Down