Skip to content

[SPARK-12006][ML][PYTHON] Fix GMM failure if initialModel is not None #9986

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
zero323 wants to merge 1 commit into from
Closed

[SPARK-12006][ML][PYTHON] Fix GMM failure if initialModel is not None #9986

zero323 wants to merge 1 commit into from

Conversation

@zero323
Copy link
Member

@zero323 zero323 commented Nov 26, 2015

If initial model passed to GMM is not empty it causes net.razorvine.pickle.PickleException. It can be fixed by converting initialModel.weights to list.

thunterdb
thunterdb reviewed Jan 5, 2016
View reviewed changes
python/pyspark/mllib/clustering.py Outdated
Copy link
Contributor

@thunterdb thunterdb Jan 5, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of doing this, I suggest you explicitly call list(initialModel.weights), it is easier to understand.

@SparkQA
Copy link

SparkQA commented Jan 5, 2016

Test build #2328 has finished for PR 9986 at commit 9fc7010.

  • This patch fails R style tests.
  • This patch does not merge cleanly.
  • This patch adds no public classes.

@zero323
Copy link
Member Author

zero323 commented Jan 6, 2016

@thunterdb Done. I am not sure if a separate test is justified. It could be better to add a second round to test_gmm.

@jkbradley
Copy link
Member

jkbradley commented Jan 6, 2016

I think the test is fine for now. LGTM

Merging with master, branch-1.6, branch-1.5, branch-1.4 (tested with old versions)

Thanks!

asfgit pushed a commit that referenced this pull request Jan 6, 2016
@zero323 @jkbradley
[SPARK-12006][ML][PYTHON] Fix GMM failure if initialModel is not None
1756819 
If initial model passed to GMM is not empty it causes `net.razorvine.pickle.PickleException`. It can be fixed by converting `initialModel.weights` to `list`.

Author: zero323 <[email protected]>

Closes #9986 from zero323/SPARK-12006.

(cherry picked from commit fcd013c)
Signed-off-by: Joseph K. Bradley <[email protected]>
asfgit pushed a commit that referenced this pull request Jan 6, 2016
@zero323 @jkbradley
[SPARK-12006][ML][PYTHON] Fix GMM failure if initialModel is not None
5d2d2dd 
If initial model passed to GMM is not empty it causes `net.razorvine.pickle.PickleException`. It can be fixed by converting `initialModel.weights` to `list`.

Author: zero323 <[email protected]>

Closes #9986 from zero323/SPARK-12006.

(cherry picked from commit fcd013c)
Signed-off-by: Joseph K. Bradley <[email protected]>
asfgit pushed a commit that referenced this pull request Jan 6, 2016
@zero323 @jkbradley
[SPARK-12006][ML][PYTHON] Fix GMM failure if initialModel is not None
bc39775 
If initial model passed to GMM is not empty it causes `net.razorvine.pickle.PickleException`. It can be fixed by converting `initialModel.weights` to `list`.

Author: zero323 <[email protected]>

Closes #9986 from zero323/SPARK-12006.

(cherry picked from commit fcd013c)
Signed-off-by: Joseph K. Bradley <[email protected]>
@asfgit asfgit closed this in fcd013c Jan 6, 2016
@zero323
Copy link
Member Author

zero323 commented Jan 7, 2016

@jkbradley I am glad to hear that. Thanks for your assistance.

@yhuai
Copy link
Contributor

yhuai commented Jan 7, 2016

seems this one was merged without a passing test?

@yhuai
Copy link
Contributor

yhuai commented Jan 7, 2016

looks like jenkins did not run the test for the latest commit.

@yhuai
Copy link
Contributor

yhuai commented Jan 7, 2016

I am reverting it from 1.4, 1.5, 1.6, and master.

@yhuai
Copy link
Contributor

yhuai commented Jan 7, 2016

Sorry. We need to make sure it passes jenkins.

@yhuai
Copy link
Contributor

yhuai commented Jan 7, 2016

@zero323 I have reverted this PR (see #10632). Can you create a new PR and remove the whitespace at the end of line self.assertAlmostEqual((gmm1.weights - gmm2.weights).sum(), 0.0)?

@zero323
Copy link
Member Author

zero323 commented Jan 7, 2016

@yhuai #10644

@jkbradley
Copy link
Member

jkbradley commented Jan 7, 2016

@yhuai I'm sorry about that! I should not merge when tired...

@yhuai
Copy link
Contributor

yhuai commented Jan 7, 2016

no worries. Looks #10644 has passed jenkins.

@zero323
Copy link
Member Author

zero323 commented Jan 7, 2016

@jkbradley, @yhuai It is my bad. Sorry for the mess.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@zero323 @SparkQA @jkbradley @yhuai @thunterdb