Skip to content

[SPARK-3826][SQL]enable hive-thriftserver to support hive-0.13.1 #2685

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
scwf wants to merge 22 commits into from
Closed

[SPARK-3826][SQL]enable hive-thriftserver to support hive-0.13.1 #2685

Changes from 1 commit
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
ae47489
update and fix conflicts
scwf Oct 14, 2014
7c66b8e
update pom according spark-2706
scwf Oct 14, 2014
c6da3ce
Merge branch 'master' of https://github.com/apache/spark into shim-th…
scwf Oct 14, 2014
dfd1c63
update run-tests to run hive-0.12.0 default now
scwf Oct 14, 2014
0bc53aa
fixed when result filed is null
scwf Oct 16, 2014
4b681f4
enable thriftserver in profile hive-0.13.1
scwf Oct 16, 2014
13afde0
fix small bug
scwf Oct 16, 2014
41f727b
revert pom changes
scwf Oct 23, 2014
f48d3a5
Merge branch 'master' of https://github.com/apache/spark into shim-th…
scwf Oct 23, 2014
f51ff4e
update and fix conflicts
scwf Oct 25, 2014
3529e98
move hive module to hive profile
scwf Oct 27, 2014
52674a4
sql/hive included since examples depend on it
scwf Oct 27, 2014
c359822
reuse getCommandProcessor in hiveshim
scwf Oct 28, 2014
bcf943f
Merge branch 'master' of https://github.com/apache/spark into shim-th…
scwf Oct 28, 2014
f7c93ae
adding build with hive 0.13 before running tests
scwf Oct 28, 2014
0d7f6cf
address comments
scwf Oct 28, 2014
8a4daf2
minor fix
scwf Oct 28, 2014
fa21d09
clean package assembly/assembly
scwf Oct 29, 2014
18fb1ff
exclude kryo in hive pom
scwf Oct 30, 2014
578234d
use new shaded hive
scwf Oct 30, 2014
f5cac74
remove local hivecontext test
scwf Oct 31, 2014
f26f3be
remove clean to save time
scwf Oct 31, 2014
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
minor fix
  • Loading branch information
@scwf
scwf committed Oct 28, 2014
commit 8a4daf2a3d9a1529eb8f6598bbb619e6a0283002
4 changes: 2 additions & 2 deletions dev/run-tests
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -153,12 +153,12 @@ CURRENT_BLOCK=$BLOCK_BUILD
# QUESTION: Why doesn't 'grep -v -e "^\[info\] Resolving"' work?
# First build with 0.12 to ensure patches do not break the hive 12 build
echo -e "q\n" \
| sbt/sbt $BUILD_MVN_PROFILE_ARGS clean hive/compile hive-thiftserver/compile \
| sbt/sbt $BUILD_MVN_PROFILE_ARGS clean hive/compile hive-thriftserver/compile \
Copy link
Contributor

@pwendell pwendell Oct 31, 2014

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I actually don't think this is compiling against Hive 0.12 right now... is it?

Copy link
Contributor Author

@scwf scwf Oct 31, 2014

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think it's against 0.12 because BUILD_MVN_PROFILE_ARGS="$SBT_MAVEN_PROFILES_ARGS -Phive -Phive-0.12.0", right?

Copy link
Contributor

@liancheng liancheng Oct 31, 2014

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It should be, BUILD_MVN_PROFILE_ARGS is defined above with -Phive-0.12.0:

  BUILD_MVN_PROFILE_ARGS="$SBT_MAVEN_PROFILES_ARGS -Phive -Phive-0.12.0"

| grep -v -e "info.*Resolving" -e "warn.*Merging" -e "info.*Including"

# Then build with default version(0.13.1) because tests based on this version
Copy link
Contributor

@marmbrus marmbrus Oct 29, 2014

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

because tests are based on this version.

echo -e "q\n" \
| sbt/sbt $SBT_MAVEN_PROFILES_ARGS -Phive clean hive/compile hive-thiftserver/compile \
| sbt/sbt $SBT_MAVEN_PROFILES_ARGS -Phive clean assembly/assembly \
Copy link
Contributor

@marmbrus marmbrus Oct 29, 2014

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lets put package back in between clean and assembly/assembly

| grep -v -e "info.*Resolving" -e "warn.*Merging" -e "info.*Including"
}

Expand Down