[SPARK-12603] [MLlib] PySpark MLlib GaussianMixtureModel should support single instance predict/predictSoft

[yanboliang]

PySpark MLlib GaussianMixtureModel should support single instance predict/predictSoft just like Scala do.

[SparkQA]

Test build #48580 has finished for PR 10552 at commit cbda57e.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[jkbradley]

These changes look fine to me, but could you please test predictSoft in Python (also to add this to the example in the doc)?

[SparkQA]

Test build #48827 has finished for PR 10552 at commit da45010.

• This patch fails PySpark unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #48845 has finished for PR 10552 at commit af22848.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[jkbradley]

I'd add a println after this for spacing.

[jkbradley]

Looks good except for the 1 comment

[SparkQA]

Test build #2338 has finished for PR 10552 at commit af22848.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #48933 has finished for PR 10552 at commit ee56c7b.

• This patch fails PySpark unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[jkbradley]

I'm finding that this unit test fails on my laptop. I'm wondering if the problem is: different default parallelism levels on different machines => clusterdata_1 has different numbers of partitions => GMM gets initialized with different samples.

Could you please change the sc.parallelize call to use 2 partitions to see if that fixes the test?

[SparkQA]

Test build #49004 has finished for PR 10552 at commit 06f4033.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[yanboliang]

@jkbradley It can produce stable result after changing sc.parallelize to use 2 partitions. Thanks!
Due to the predictSoft output may be negligibly small that we can not use 0.000... to match equality, so I use abs(softPredicted[1] - 0.0) < 0.001 in doctest.

[jkbradley]

That looks good. I also just noticed: Could you please update the docs for Python predict, predictSoft to say they work on RDDs and single vectors? That should be it.

[SparkQA]

Test build #49132 has finished for PR 10552 at commit f6717a8.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[jkbradley]

LGTM
Merging with master
Thanks for the PR!