Skip to content

[java8API] SPARK-964 Investigate the potential for using JDK 8 lambda expressions for the Java/Scala APIs #17

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
ScrapCodes wants to merge 9 commits into from
Closed

[java8API] SPARK-964 Investigate the potential for using JDK 8 lambda expressions for the Java/Scala APIs #17

Changes from 1 commit
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
c33dc2c
SPARK-964, Java 8 API Support.
ScrapCodes Feb 24, 2014
4ab87d3
Review feedback on the pr
ScrapCodes Feb 25, 2014
31d4cd6
Maven build to support -Pjava8-tests flag.
ScrapCodes Feb 25, 2014
35d8d79
Specified java 8 building in the docs
ScrapCodes Feb 26, 2014
26eb3f6
Patrick's comments on PR.
ScrapCodes Mar 3, 2014
80a13e8
Used fake class tag syntax
ScrapCodes Mar 3, 2014
673f7ac
Added support for -java-home as well
ScrapCodes Mar 3, 2014
85a954e
Nit. import orderings.
ScrapCodes Mar 3, 2014
95850e6
Some doc improvements and build changes to the Java 8 patch.
pwendell Mar 3, 2014
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Specified java 8 building in the docs
  • Loading branch information
@ScrapCodes
ScrapCodes committed Mar 3, 2014
commit 35d8d79e4f1ccb6491b81fd670043e2b6c60a815
10 changes: 10 additions & 0 deletions docs/building-with-maven.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,8 @@ If you don't run this, you may see errors like the following:

You can fix this by setting the `MAVEN_OPTS` variable as discussed before.

*Note: For Java 1.8 and above this step is not required.*

## Specifying the Hadoop version ##

Because HDFS is not protocol-compatible across versions, if you want to read from HDFS, you'll need to build Spark against the specific HDFS version in your environment. You can do this through the "hadoop.version" property. If unset, Spark will build against Hadoop 1.0.4 by default.
Expand Down Expand Up @@ -76,3 +78,11 @@ The maven build includes support for building a Debian package containing the as
$ mvn -Pdeb -DskipTests clean package

The debian package can then be found under assembly/target. We added the short commit hash to the file name so that we can distinguish individual packages built for SNAPSHOT versions.

## Running java 8 test suites.

Running only java 8 tests and nothing else.

$ mvn test -DskipTests -Pjava8-tests

Java 8 tests are run when -Pjava8-tests profile is enabled, they will run inspite of -DskipTests. For these tests to run java 8 should be installed on the system running the tests.
Copy link
Contributor

@pwendell pwendell Mar 1, 2014

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

in spite of

Copy link
Contributor

@pwendell pwendell Mar 1, 2014

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It might also be good to explain that JAVA_HOME needs to be set to the JDK 8 installation:

"For these tests to run, JAVA_HOME must point to an installation of JDK 8." (or something similar)

Copy link
Contributor

@pwendell pwendell Mar 1, 2014

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually I think it's more specifically the following:

"For these tests to run your system must have a JDK 8 installation. If you have JDK 8 installed but it is not the system default, you can set JAVA_HOME to point to JDK 8 before running the tests."