Skip to content

SPARK-1311 Map side distinct in collect vertex ids from edges graphx #21

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
holdenk wants to merge 6 commits into from
+22 −5
Closed

SPARK-1311 Map side distinct in collect vertex ids from edges graphx #21

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
2added7
Make it distinct
holdenk Feb 27, 2014
3219bff
Use a hash set for unique in graphx instead
holdenk Mar 8, 2014
4eb59af
Fix typo
holdenk Mar 8, 2014
e055a31
Update the OpenHashSet comment to make sense (there is no constant EX…
holdenk Mar 8, 2014
238a4d8
Fix the tests I added and the comment
holdenk Mar 8, 2014
42d9522
traversableonce you drive me crazy one again
holdenk Mar 8, 2014
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Fix the tests I added and the comment
  • Loading branch information
@holdenk
holdenk committed Mar 8, 2014
commit 238a4d8a78a086b3a13c737b4c97e3e008377b55
4 changes: 2 additions & 2 deletions core/src/main/scala/org/apache/spark/util/collection/OpenHashSet.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -113,8 +113,8 @@ class OpenHashSet[@specialized(Long, Int) T: ClassTag](
* Add an element to the set. This one differs from add in that it doesn't trigger rehashing.
* The caller is responsible for calling rehashIfNeeded.
*
* Use (retval & POSITION_MASK) to get the actual position, and
* (retval & NONEXISTENCE_MASK) != 0 for prior existence.
* Use (retval & _mask) to get the actual position, and
* (retval & NONEXISTENCE_MASK) == 0 for prior existence.
*
* @return The position where the key is placed, plus the highest order bit is set if the key
* exists previously.
Expand Down
4 changes: 2 additions & 2 deletions core/src/test/scala/org/apache/spark/util/collection/OpenHashSetSuite.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -74,8 +74,8 @@ class OpenHashSetSuite extends FunSuite with ShouldMatchers {
assert(set.contains(999))
assert(!set.contains(10000))

assert(set.addWithoutResize(50) & set.NONEXISTENCE_MASK != 0)
assert(set.addWithoutResize(10000) & set.NONEXISTENCE_MASK === 0)
assert((set.addWithoutResize(50) & OpenHashSet.NONEXISTENCE_MASK) === 0)
assert((set.addWithoutResize(10000) & OpenHashSet.NONEXISTENCE_MASK) != 0)
}

test("primitive long") {
Expand Down