Skip to content

[WIP][SPARK-29108][SQL] Add new module sql/thriftserver and add v11 thrift protocol #26221

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
AngersZhuuuu wants to merge 12 commits into from
Closed

[WIP][SPARK-29108][SQL] Add new module sql/thriftserver and add v11 thrift protocol #26221

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
8b56400
Add new module and add v11 thrift protocol
AngersZhuuuu Oct 22, 2019
e30f686
add pom lincense
AngersZhuuuu Oct 23, 2019
4a50bc9
update code folder name
AngersZhuuuu Oct 23, 2019
7ffdc3b
all like hive
AngersZhuuuu Oct 23, 2019
b74d5e0
remove py/cpp/r/php
AngersZhuuuu Oct 23, 2019
fc2648f
Maven generate thrift source code
wangyum Oct 24, 2019
5365dcf
org.apache.thrift.tools:maven-thrift-plugin -> org.apache.thrift:thri…
wangyum Oct 25, 2019
95d8137
save some basic code
AngersZhuuuu Oct 26, 2019
4dc5c7e
Merge pull request #2 from wangyum/SPARK-29108
AngersZhuuuu Oct 27, 2019
24ce6d4
Merge branch 'SPARK-29018-V11' of https://github.com/AngersZhuuuu/spa…
AngersZhuuuu Oct 27, 2019
5efe8cb
Revert "save some basic code"
AngersZhuuuu Oct 27, 2019
6f7d48a
Update TCLIService.thrift
AngersZhuuuu Oct 27, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
remove py/cpp/r/php
  • Loading branch information
@AngersZhuuuu
AngersZhuuuu committed Oct 23, 2019
commit b74d5e08f9f22f41bc57031a19b691e61d8a91de
7 changes: 6 additions & 1 deletion pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -240,6 +240,11 @@
<spark.test.home>${session.executionRootDirectory}</spark.test.home>

<CodeCacheSize>1g</CodeCacheSize>

<!-- Thrift properties -->
<thrift.home>you-must-set-this-to-run-thrift</thrift.home>
Copy link
Contributor

@juliuszsompolski juliuszsompolski Oct 24, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does it need to be set, or can Maven find thrift from PATH? Or even better, do the generation via libthrift from maven central itself?

<thrift.gen.dir>${basedir}/src/gen/thrift</thrift.gen.dir>
<thrift.args>-I ${thrift.home} --gen java:beans,hashcode,generated_annotations=undated</thrift.args>
Copy link
Contributor

@juliuszsompolski juliuszsompolski Oct 24, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(just asking, I don't know maven): should this be set in global pom, or in thriftserver pom? How can this be used?

If it requires setting it manually in your pom, and the generated files are committed anyway, they I would not do it.

Copy link
Contributor Author

@AngersZhuuuu AngersZhuuuu Oct 24, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(just asking, I don't know maven): should this be set in global pom, or in thriftserver pom? How can this be used?

If it requires setting it manually in your pom, and the generated files are committed anyway, they I would not do it.

Current same as hive, make it like antlr4 is best but I may need some help... since I am particularly good at maven various plugin, I am learning the hive's approach.

Copy link
Member

@wangyum wangyum Oct 24, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@juliuszsompolski Please see this PR: AngersZhuuuu#2

</properties>
<repositories>
<repository>
Expand Down Expand Up @@ -3100,7 +3105,7 @@
<target>
<taskdef name="for" classname="net.sf.antcontrib.logic.ForTask"
classpathref="maven.plugin.classpath" />
<property name="thrift.args" value="-I ${thrift.home} --gen java:beans,hashcode,generated_annotations=undated --gen cpp --gen php --gen py --gen rb"/>
<property name="thrift.args" value="-I ${thrift.home} --gen java:beans,hashcode,generated_annotations=undated"/>
<property name="thrift.gen.dir" value="${basedir}/src/gen/thrift"/>
<delete dir="${thrift.gen.dir}"/>
<mkdir dir="${thrift.gen.dir}"/>
Expand Down
3 changes: 0 additions & 3 deletions sql/thriftserver/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,3 @@
Thrift commands to generate files from TCLIService.thrift:
--------------------
thrift --gen java:beans,hashcode -o src/gen/thrift if/TCLIService.thrift
thrift --gen cpp -o src/gen/thrift if/TCLIService.thrift
thrift --gen py -o src/gen/thrift if/TCLIService.thrift
thrift --gen rb -o src/gen/thrift if/TCLIService.thrift
4 changes: 0 additions & 4 deletions sql/thriftserver/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,10 +32,6 @@
<url>http://spark.apache.org/</url>
<properties>
<sbt.project.name>thriftserver</sbt.project.name>
<!-- Thrift properties -->
<thrift.home>you-must-set-this-to-run-thrift</thrift.home>
<thrift.gen.dir>${basedir}/src/gen/thrift</thrift.gen.dir>
<thrift.args>-I ${thrift.home} --gen java:beans,hashcode,generated_annotations=undated --gen cpp --gen php --gen py --gen rb</thrift.args>
</properties>

<dependencies>
Expand Down
Loading