Skip to content

[SPARK-28527][SQL][TEST] Re-run all the tests in SQLQueryTestSuite via Thrift Server #25567

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wangyum wants to merge 11 commits into from
Closed

[SPARK-28527][SQL][TEST] Re-run all the tests in SQLQueryTestSuite via Thrift Server #25567

Changes from 1 commit
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
699b4db
Directly re-run all the tests in SQLQueryTestSuite via Thrift Server
wangyum Aug 7, 2019
908cdc3
avoid test with ConfigSets
wangyum Aug 7, 2019
66e9b25
Merge remote-tracking branch 'upstream/master' into SPARK-28527
wangyum Aug 8, 2019
3f21189
Add ThriftServerQueryTestSuite to RunInTheirOwnDedicatedJvm
wangyum Aug 8, 2019
5802888
Move startThriftServer from beforeEach to beforeAll
wangyum Aug 13, 2019
02c6911
Merge remote-tracking branch 'upstream/master' into SPARK-28527
wangyum Aug 20, 2019
2823f34
Maven can not get the correct baseResourcePath
wangyum Aug 20, 2019
4845088
Fix pgSQL/text.sql
wangyum Aug 20, 2019
beafc57
Workaround Maven can't get correct resource directory when resources …
wangyum Aug 23, 2019
4aa5ee2
Fix no viable alternative:
wangyum Aug 24, 2019
2254809
Use Spark home when it detects 'sql-tests' directory (#27)
HyukjinKwon Aug 25, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Maven can not get the correct baseResourcePath
  • Loading branch information
@wangyum
wangyum committed Aug 20, 2019
commit 2823f3472701f1e002a6795dab5e15b4373a58cb
32 changes: 19 additions & 13 deletions ...er/src/test/scala/org/apache/spark/sql/hive/thriftserver/ThriftServerQueryTestSuite.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@ import org.apache.spark.sql.types._

/**
* Re-run all the tests in SQLQueryTestSuite via Thrift Server.
* Note that this TestSuite does not support maven.
*
* TODO:
* 1. Support UDF testing.
Expand All @@ -60,7 +61,7 @@ class ThriftServerQueryTestSuite extends SQLQueryTestSuite {
case cause: Throwable =>
throw cause
}.get
logInfo(s"HiveThriftServer2 started successfully")
logInfo("HiveThriftServer2 started successfully")
}

override def afterAll(): Unit = {
Expand Down Expand Up @@ -135,7 +136,7 @@ class ThriftServerQueryTestSuite extends SQLQueryTestSuite {
// each query has 3 segments, plus the header
assert(segments.size == outputs.size * 3 + 1,
s"Expected ${outputs.size * 3 + 1} blocks in result file but got ${segments.size}. " +
s"Try regenerate the result files.")
"Try regenerate the result files.")
Seq.tabulate(outputs.size) { i =>
val sql = segments(i * 3 + 1).trim
val originalOut = segments(i * 3 + 3)
Expand Down Expand Up @@ -204,18 +205,23 @@ class ThriftServerQueryTestSuite extends SQLQueryTestSuite {
}

override def listTestCases(): Seq[TestCase] = {
listFilesRecursively(new File(inputFilePath)).flatMap { file =>
val resultFile = file.getAbsolutePath.replace(inputFilePath, goldenFilePath) + ".out"
val absPath = file.getAbsolutePath
val testCaseName = absPath.stripPrefix(inputFilePath).stripPrefix(File.separator)

if (file.getAbsolutePath.startsWith(s"$inputFilePath${File.separator}udf")) {
Seq.empty
} else if (file.getAbsolutePath.startsWith(s"$inputFilePath${File.separator}pgSQL")) {
PgSQLTestCase(testCaseName, absPath, resultFile) :: Nil
} else {
RegularTestCase(testCaseName, absPath, resultFile) :: Nil
// Maven can not get the correct baseResourcePath
if (baseResourcePath.exists()) {
listFilesRecursively(new File(inputFilePath)).flatMap { file =>
val resultFile = file.getAbsolutePath.replace(inputFilePath, goldenFilePath) + ".out"
val absPath = file.getAbsolutePath
val testCaseName = absPath.stripPrefix(inputFilePath).stripPrefix(File.separator)

if (file.getAbsolutePath.startsWith(s"$inputFilePath${File.separator}udf")) {
Seq.empty
} else if (file.getAbsolutePath.startsWith(s"$inputFilePath${File.separator}pgSQL")) {
PgSQLTestCase(testCaseName, absPath, resultFile) :: Nil
} else {
RegularTestCase(testCaseName, absPath, resultFile) :: Nil
}
}
} else {
Seq.empty[TestCase]
}
}

Expand Down