Skip to content
Closed
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Added support for -java-home as well
  • Loading branch information
ScrapCodes committed Mar 3, 2014
commit 673f7ac9e8855e3be16e2e955d0c01d1b187073a
12 changes: 7 additions & 5 deletions extras/java8-tests/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,12 +2,14 @@

These tests are bundled with spark and run if you have java 8 installed as system default or your `JAVA_HOME` points to a java 8(or higher) installation. `JAVA_HOME` is preferred to system default jdk installation. Since these tests require jdk 8 or higher, they defined to be optional to run in the build system.
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

they defined -> they are defined


For sbt users, it automatically detects the presence of java 8 based on either `JAVA_HOME` environment variable or default installed jdk and run these tests.
* For sbt users, it automatically detects the presence of java 8 based on either `JAVA_HOME` environment variable or default installed jdk and run these tests. It takes highest precednce, if java home is passed as follows.

For maven users,
`$ sbt/sbt -java-home "/path/to/jdk1.8.0"`

This automatic detection is not possible and thus user has to ensure that either `JAVA_HOME` environment variable or default installed jdk points to jdk 8.
* For maven users,

`$ mvn install -Pjava8-tests`
This automatic detection is not possible and thus user has to ensure that either `JAVA_HOME` environment variable or default installed jdk points to jdk 8.

Above command can only be run from project root directory since this module depends on both core and test-jars of core and streaming. These jars are installed first time the above command is run as java8-tests profile enables local publishing of test-jar artifacts as well. Once these artifacts are published then these tests can be run from this module's directory as well.
`$ mvn install -Pjava8-tests`

Above command can only be run from project root directory since this module depends on both core and test-jars of core and streaming. These jars are installed first time the above command is run as java8-tests profile enables local publishing of test-jar artifacts as well. Once these artifacts are published then these tests can be run from this module's directory as well.
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,19 @@ public void tearDown() {
System.clearProperty("spark.driver.port");
}

@Test
public void foreachWithAnonymousClass() {
foreachCalls = 0;
JavaRDD<String> rdd = sc.parallelize(Arrays.asList("Hello", "World"));
rdd.foreach(new VoidFunction<String>() {
@Override
public void call(String s) {
foreachCalls++;
}
});
Assert.assertEquals(2, foreachCalls);
}

@Test
public void foreach() {
foreachCalls = 0;
Expand Down
4 changes: 1 addition & 3 deletions project/SparkBuild.scala
Original file line number Diff line number Diff line change
Expand Up @@ -98,8 +98,6 @@ object SparkBuild extends Build {
lazy val java8Tests = Project("java8-tests", file("extras/java8-tests"), settings = java8TestsSettings).
dependsOn(core) dependsOn(streaming % "compile->compile;test->test")

val javaHomeEnv = Properties.envOrNone("JAVA_HOME").map(file)

// Conditionally include the yarn sub-project
lazy val yarnAlpha = Project("yarn-alpha", file("yarn/alpha"), settings = yarnAlphaSettings) dependsOn(core)
lazy val yarn = Project("yarn", file("yarn/stable"), settings = yarnSettings) dependsOn(core)
Expand Down Expand Up @@ -143,7 +141,7 @@ object SparkBuild extends Build {
javacOptions := Seq("-target", JAVAC_JVM_VERSION, "-source", JAVAC_JVM_VERSION),
unmanagedJars in Compile <<= baseDirectory map { base => (base / "lib" ** "*.jar").classpath },
retrieveManaged := true,
if (javaHomeEnv.isDefined) javaHome := javaHomeEnv else javaHome <<= javaHome in Compile,
javaHome := Properties.envOrNone("JAVA_HOME").map(file),
// This is to add convenience of enabling sbt -Dsbt.offline=true for making the build offline.
offline := "true".equalsIgnoreCase(sys.props("sbt.offline")),
retrievePattern := "[type]s/[artifact](-[revision])(-[classifier]).[ext]",
Expand Down
2 changes: 1 addition & 1 deletion sbt/sbt-launch-lib.bash
Original file line number Diff line number Diff line change
Expand Up @@ -137,7 +137,7 @@ process_args () {

-sbt-jar) require_arg path "$1" "$2" && sbt_jar="$2" && shift 2 ;;
-sbt-version) require_arg version "$1" "$2" && sbt_version="$2" && shift 2 ;;
-java-home) require_arg path "$1" "$2" && java_cmd="$2/bin/java" && shift 2 ;;
-java-home) require_arg path "$1" "$2" && java_cmd="$2/bin/java" && export JAVA_HOME=$2 && shift 2 ;;

-D*) addJava "$1" && shift ;;
-J*) addJava "${1:2}" && shift ;;
Expand Down