Skip to content

[SPARK-26794][SQL]SparkSession enableHiveSupport does not point to hive but in-memory while the SparkContext exists #23709

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
yaooqinn wants to merge 15 commits into from
Closed

[SPARK-26794][SQL]SparkSession enableHiveSupport does not point to hive but in-memory while the SparkContext exists #23709

Changes from 1 commit
Commits
Show all changes
15 commits
Select commit Hold shift + click to select a range
275e998
fix #SPARK-26794 SparkSession enableHiveSupport does not point to hiv…
yaooqinn Jan 31, 2019
e1eafe7
fix compile
yaooqinn Jan 31, 2019
4a1a18b
add ut
yaooqinn Jan 31, 2019
62d2aa3
clone conf and apply all initiate options to them
yaooqinn Feb 1, 2019
8004c19
typo & fix ut
yaooqinn Feb 1, 2019
10ebff7
not import map
yaooqinn Feb 1, 2019
f5c15e0
naming
yaooqinn Feb 1, 2019
d57226a
set conf to lazy val
yaooqinn Feb 1, 2019
0c07ff1
fix ut
yaooqinn Feb 1, 2019
35b3f17
add some comments
yaooqinn Feb 1, 2019
0cac5e6
add more ut
yaooqinn Feb 1, 2019
92fec4d
fix ut
yaooqinn Feb 2, 2019
391e14c
mv confs behind warehousePath
yaooqinn Feb 12, 2019
eff569b
meaningful titles for uts
yaooqinn Feb 12, 2019
748821c
rm one ut
yaooqinn Feb 13, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
fix ut
  • Loading branch information
@yaooqinn
yaooqinn committed Feb 1, 2019
commit 0c07ff13bf88a15d1bec81e39db4573c38649ed9
26 changes: 17 additions & 9 deletions sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveSharedStateSuite.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,15 +23,23 @@ import org.apache.spark.sql.SparkSession
class HiveSharedStateSuite extends SparkFunSuite {

test("the catalog should be determined at the very first") {
val conf = new SparkConf().setMaster("local").setAppName("SharedState Test")
val sc = SparkContext.getOrCreate(conf)
val ss = SparkSession.builder().enableHiveSupport().getOrCreate()
assert(ss.sharedState.externalCatalog.unwrapped.getClass.getName ===
"org.apache.spark.sql.hive.HiveExternalCatalog", "The catalog should be hive ")
SparkContext.getActive.foreach(_.stop())
var sc: SparkContext = null
try {
val conf = new SparkConf().setMaster("local").setAppName("SharedState Test")
sc = new SparkContext(conf)
val ss = SparkSession.builder().enableHiveSupport().getOrCreate()
assert(ss.sharedState.externalCatalog.unwrapped.getClass.getName ===
"org.apache.spark.sql.hive.HiveExternalCatalog", "The catalog should be hive ")

val ss2 = SparkSession.builder().getOrCreate()
assert(ss2.sharedState.externalCatalog.unwrapped.getClass.getName ===
"org.apache.spark.sql.hive.HiveExternalCatalog",
"The catalog should be shared across sessions")
val ss2 = SparkSession.builder().getOrCreate()
assert(ss2.sharedState.externalCatalog.unwrapped.getClass.getName ===
"org.apache.spark.sql.hive.HiveExternalCatalog",
"The catalog should be shared across sessions")
} finally {
if (sc != null && !sc.isStopped) {
sc.stop()
}
}
}
}