Skip to content
Closed
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Add Java 8 API test
  • Loading branch information
holdenk committed Mar 15, 2014
commit cc4050d86032d6da2f9662bb8a2595567a994210
Original file line number Diff line number Diff line change
Expand Up @@ -245,6 +245,24 @@ public void mapPartitions() {
Assert.assertEquals("[3, 7]", partitionSums.collect().toString());
}

@Test
public void mapPartitionsWithIndex() {
JavaRDD<Integer> rdd = sc.parallelize(Arrays.asList(1, 2, 3, 4), 2);
JavaRDD<Integer> partitionSums = rdd.mapPartitionsWithIndex((start, iter) -> {
List<Integer> list = new ArrayList<Integer>();
int sum = 0;
int pos = start;
while (iter.hasNext()) {
sum += (pos* iter.next());
pos += 1;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indents seem messed up here, should both be 2 spaces

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also space before *

}
return list.iterator();
});
Assert.assertEquals(0, rddByIndex.first().intValue());
Integer[] values = {0, 2, 6, 12, 20};
Assert.assertEquals(Arrays.asList(values), rddByIndex.collect());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually this test is just broken as is, there's no rddByIndex variable. You have to run it on Java 8, otherwise SBT will not build this project.

}

@Test
public void sequenceFile() {
File tempDir = Files.createTempDir();
Expand Down