Skip to content

[SPARK-23808][SQL] Set default Spark session in test-only spark sessions. #20926

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
jose-torres wants to merge 6 commits into from
Closed

[SPARK-23808][SQL] Set default Spark session in test-only spark sessions. #20926

Changes from 1 commit
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
ffa816e
set defaults and add test
jose-torres Mar 28, 2018
d0988f7
set in shared session and clean up
jose-torres Mar 29, 2018
c6613ff
Revert "set in shared session and clean up"
jose-torres Mar 29, 2018
7be16a9
close spark session in test
jose-torres Mar 29, 2018
851a5ef
don't double set default session in hive
jose-torres Mar 29, 2018
d9c996f
leave todo in hive
jose-torres Mar 29, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
don't double set default session in hive
  • Loading branch information
@jose-torres
jose-torres committed Mar 29, 2018
commit 851a5efa87a9f10843ec9d45437d6a5d94cc0816
2 changes: 0 additions & 2 deletions sql/hive/src/main/scala/org/apache/spark/sql/hive/test/TestHive.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -175,8 +175,6 @@ private[hive] class TestHiveSparkSession(
loadTestTables)
}

SparkSession.setDefaultSession(this)
Copy link
Member

@gatorsmile gatorsmile Mar 29, 2018 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where did we set this for TestHive?

Copy link
Contributor Author

@jose-torres jose-torres Mar 29, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Many Hive tests both call SparkSession.getOrCreate() and create a TestHiveSparkSession. To be clear, this isn't an optimization - the tests do not pass unless I remove this line.

Copy link
Contributor Author

@jose-torres jose-torres Mar 29, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As discussed offline, we will remove the TestHive* changes and replace with a TODO.


{ // set the metastore temporary configuration
val metastoreTempConf = HiveUtils.newTemporaryConfiguration(useInMemoryDerby = false) ++ Map(
ConfVars.METASTORE_INTEGER_JDO_PUSHDOWN.varname -> "true",
Expand Down