-
Notifications
You must be signed in to change notification settings - Fork 29k
[WIP][SPARK-29108][SQL] Add new module sql/thriftserver with all code and UT #26340
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Closed
AngersZhuuuu
wants to merge
58
commits into
apache:master
from
AngersZhuuuu:SPARK-29018-V11-STEP4-ADD-TEST
Closed
Changes from 1 commit
Commits
Show all changes
58 commits
Select commit
Hold shift + click to select a range
8b56400
Add new module and add v11 thrift protocol
AngersZhuuuu e30f686
add pom lincense
AngersZhuuuu 4a50bc9
update code folder name
AngersZhuuuu 7ffdc3b
all like hive
AngersZhuuuu b74d5e0
remove py/cpp/r/php
AngersZhuuuu fc2648f
Maven generate thrift source code
wangyum 5365dcf
org.apache.thrift.tools:maven-thrift-plugin -> org.apache.thrift:thri…
wangyum 95d8137
save some basic code
AngersZhuuuu 4dc5c7e
Merge pull request #2 from wangyum/SPARK-29108
AngersZhuuuu 24ce6d4
Merge branch 'SPARK-29018-V11' of https://github.com/AngersZhuuuu/spa…
AngersZhuuuu 5efe8cb
Revert "save some basic code"
AngersZhuuuu 6f7d48a
Update TCLIService.thrift
AngersZhuuuu cf24306
Add basic data
AngersZhuuuu 24fce6b
save code
AngersZhuuuu bc36bdf
change
AngersZhuuuu c438673
format code
AngersZhuuuu a7ec8b0
format code
AngersZhuuuu 5cf7eb0
Update SparkSQLEnv.scala
AngersZhuuuu 0023bcb
fix logger conflict
AngersZhuuuu 723450b
fix scala style
AngersZhuuuu 3eb7672
start with execution hive
AngersZhuuuu 5aa4d9d
format code add file header
AngersZhuuuu 782d36b
FOR BUILD spark-thriftserver
AngersZhuuuu f2b5346
add UT class
AngersZhuuuu a14a9e9
fix UT case, remove can't supported UT
AngersZhuuuu e1bb6e1
fix for UT
AngersZhuuuu 74fb240
fix UT
AngersZhuuuu 2f90ed9
fix for UT
AngersZhuuuu cd92f3f
fix return error
AngersZhuuuu 60dc24f
fix client convert row map
AngersZhuuuu 409a1a3
add UT of TCLIServiceClient
AngersZhuuuu 5771a9b
fix java code style
AngersZhuuuu 77578c3
add processGlobalInitFile
AngersZhuuuu 4f7cbac
Update ThriftServerSessionImpl.scala
AngersZhuuuu d93ab72
Keep type in scala and remove unused type
AngersZhuuuu ac9ad54
basic service to java
AngersZhuuuu 795ada1
Handle to java
AngersZhuuuu 63076d3
Operation Type State Status to java
AngersZhuuuu 000443a
Fetch type and orientation to java
AngersZhuuuu 5de2bf9
PatternOrIdentifier to java
AngersZhuuuu a6b0ed1
Interface to Java
AngersZhuuuu 2e6d221
fix bug
AngersZhuuuu 158b298
remove setApplicationName
AngersZhuuuu 745ca60
Session Basic Class to java
AngersZhuuuu f62e577
auth and thrift service to java
AngersZhuuuu 5fae842
Update ColumnBasedSet.scala
AngersZhuuuu fe39db0
Update RowBasedSet.scala
AngersZhuuuu 1c303ba
Update type map
AngersZhuuuu 9a32af8
make RowSetFactory clean
AngersZhuuuu 287c6be
update
AngersZhuuuu f6a7736
format code
AngersZhuuuu f8b7351
add thrift file
AngersZhuuuu 272ba3c
not check current java code now
AngersZhuuuu 0de3191
remove import jdk.tools dependency
AngersZhuuuu 6374f42
follow comment
AngersZhuuuu f464773
Merge branch 'master' into SPARK-29018-V11-STEP4-ADD-TEST
AngersZhuuuu 13da926
fix for `mvn install`
AngersZhuuuu d0de49f
fix for `mvn install`
AngersZhuuuu File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
format code
- Loading branch information
commit c4386731b43c84bf2508273ef5ff7c3c41de62cd
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -37,11 +37,9 @@ import org.apache.spark.sql.thriftserver.cli.thrift._ | |
| import org.apache.spark.sql.thriftserver.server.SparkThriftServer | ||
| import org.apache.spark.sql.types.StructType | ||
|
|
||
| class CLIService(hiveServer2: SparkThriftServer, sqlContext: SQLContext) | ||
| private[thriftserver] class CLIService(hiveServer2: SparkThriftServer, sqlContext: SQLContext) | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. what are the changes / could it stay in Java? |
||
| extends CompositeService(classOf[CLIService].getSimpleName) | ||
| with ICLIService with Logging { | ||
|
|
||
| import CLIService._ | ||
| with ICLIService with Logging { | ||
|
|
||
| private var hiveConf: HiveConf = null | ||
| private var sessionManager: SessionManager = null | ||
|
|
@@ -100,23 +98,7 @@ class CLIService(hiveServer2: SparkThriftServer, sqlContext: SQLContext) | |
|
|
||
| def getHttpUGI: UserGroupInformation = this.httpUGI | ||
|
|
||
| def openSession(protocol: TProtocolVersion, | ||
| username: String, | ||
| password: String, | ||
| configuration: Predef.Map[String, String]): SessionHandle = { | ||
| val sessionHandle: SessionHandle = | ||
| sessionManager.openSession(protocol, | ||
| username, | ||
| password, | ||
| null, | ||
| configuration, | ||
| false, | ||
| null) | ||
| logDebug(sessionHandle + ": openSession()") | ||
| sessionHandle | ||
| } | ||
|
|
||
| def openSession(protocol: TProtocolVersion, | ||
| override def openSession(protocol: TProtocolVersion, | ||
| username: String, | ||
| password: String, | ||
| ipAddress: String, | ||
|
|
@@ -133,24 +115,7 @@ class CLIService(hiveServer2: SparkThriftServer, sqlContext: SQLContext) | |
| sessionHandle | ||
| } | ||
|
|
||
| def openSessionWithImpersonation(protocol: TProtocolVersion, | ||
| username: String, | ||
| password: String, | ||
| configuration: Predef.Map[String, String], | ||
| delegationToken: String): SessionHandle = { | ||
| val sessionHandle = | ||
| sessionManager.openSession(protocol, | ||
| username, | ||
| password, | ||
| null, | ||
| configuration, | ||
| true, | ||
| delegationToken) | ||
| logDebug(sessionHandle + ": openSessionWithImpersonation()") | ||
| sessionHandle | ||
| } | ||
|
|
||
| def openSessionWithImpersonation(protocol: TProtocolVersion, | ||
| override def openSessionWithImpersonation(protocol: TProtocolVersion, | ||
| username: String, | ||
| password: String, | ||
| ipAddress: String, | ||
|
|
@@ -168,38 +133,6 @@ class CLIService(hiveServer2: SparkThriftServer, sqlContext: SQLContext) | |
| sessionHandle | ||
| } | ||
|
|
||
| override def openSession(username: String, | ||
| password: String, | ||
| configuration: Predef.Map[String, String]): SessionHandle = { | ||
| val sessionHandle = | ||
| sessionManager.openSession( | ||
| SERVER_VERSION, | ||
| username, | ||
| password, | ||
| null, | ||
| configuration, | ||
| false, | ||
| null) | ||
| logDebug(sessionHandle + ": openSession()") | ||
| sessionHandle | ||
| } | ||
|
|
||
| override def openSessionWithImpersonation(username: String, | ||
| password: String, | ||
| configuration: Predef.Map[String, String], | ||
| delegationToken: String): SessionHandle = { | ||
| val sessionHandle = | ||
| sessionManager.openSession(SERVER_VERSION, | ||
| username, | ||
| password, | ||
| null, | ||
| configuration, | ||
| true, | ||
| delegationToken) | ||
| logDebug(sessionHandle + ": openSessionWithImpersonation()") | ||
| sessionHandle | ||
| } | ||
|
|
||
| override def closeSession(sessionHandle: SessionHandle): Unit = { | ||
| sessionManager.closeSession(sessionHandle) | ||
| logDebug(sessionHandle + ": closeSession()") | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.