Skip to content

[SPARK-24590][BUILD] Make Jenkins tests passed with hadoop 3 profile #21588

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
HyukjinKwon wants to merge 7 commits into from
Closed

[SPARK-24590][BUILD] Make Jenkins tests passed with hadoop 3 profile #21588

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
4585d2d
Make Jenkins tests passed with hadoop 3 profile
HyukjinKwon May 28, 2018
fe8bc03
Add Hadoop version assume for Hive_2_1_DDLSuite.scala
HyukjinKwon Jun 19, 2018
dd04f18
Skip conditionally
HyukjinKwon Jun 19, 2018
be6e288
Lazy HiveClient
HyukjinKwon Jun 19, 2018
c4c82ba
Fix all other failures
HyukjinKwon Jun 20, 2018
b9eeef3
Revert the changes needed for testing
HyukjinKwon Jun 20, 2018
a011e50
typo
HyukjinKwon Sep 27, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Revert the changes needed for testing
  • Loading branch information
@HyukjinKwon
HyukjinKwon committed Sep 27, 2018
commit b9eeef3ed9a97a8edc7bde0150e2851b6996ef45
4 changes: 2 additions & 2 deletions dev/run-tests.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -305,8 +305,8 @@ def get_hadoop_profiles(hadoop_version):
"""

sbt_maven_hadoop_profiles = {
"hadoop2.6": ["-Phadoop-3.1"],
"hadoop2.7": ["-Phadoop-3.1"],
"hadoop2.6": ["-Phadoop-2.6"],
"hadoop2.7": ["-Phadoop-2.7"],
}

if hadoop_version in sbt_maven_hadoop_profiles:
Expand Down
2 changes: 1 addition & 1 deletion pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -124,7 +124,7 @@
<flume.version>1.6.0</flume.version>
<zookeeper.version>3.4.6</zookeeper.version>
<curator.version>2.6.0</curator.version>
<hive.group>com.github.hyukjinkwon</hive.group>
<hive.group>org.spark-project.hive</hive.group>
<!-- Version used in Maven Hive dependency -->
<hive.version>1.2.1.spark2</hive.version>
<!-- Version used for internal directory structure -->
Expand Down
2 changes: 1 addition & 1 deletion sql/hive/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -121,7 +121,7 @@
<artifactId>avro</artifactId>
</dependency>
<!-- use the build matching the hadoop api of avro-mapred (i.e. no classifier for hadoop 1 API,
hadoop2 classifier for hadoop 2 API. avro-mapred is a dependency of com.github.hyukjinkwon:hive-serde -->
hadoop2 classifier for hadoop 2 API. avro-mapred is a dependency of org.spark-project.hive:hive-serde -->
<dependency>
<groupId>org.apache.avro</groupId>
<artifactId>avro-mapred</artifactId>
Expand Down