Skip to content

[SPARK-29379][SQL]SHOW FUNCTIONS show '!=', '<>' , 'between', 'case' #26053

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
AngersZhuuuu wants to merge 14 commits into from
Closed

[SPARK-29379][SQL]SHOW FUNCTIONS show '!=', '<>' , 'between', 'case' #26053

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
911007a
fix show functions
AngersZhuuuu Oct 8, 2019
ba22b62
fix bug
AngersZhuuuu Oct 8, 2019
074ee0e
fit SYSTEM/USER
AngersZhuuuu Oct 8, 2019
9b8d63e
fix UT
AngersZhuuuu Oct 8, 2019
b066088
follow comment
AngersZhuuuu Oct 9, 2019
3d6c85d
follow comment
AngersZhuuuu Oct 16, 2019
85556a6
follow comment
AngersZhuuuu Oct 16, 2019
22b3487
follow comment
AngersZhuuuu Oct 16, 2019
60cd8a8
Update SQLQuerySuite.scala
AngersZhuuuu Oct 16, 2019
522193c
Revert "Update SQLQuerySuite.scala"
AngersZhuuuu Oct 16, 2019
a285290
Update SQLQuerySuite.scala
AngersZhuuuu Oct 16, 2019
9f68ead
remove empty line
AngersZhuuuu Oct 16, 2019
bc04f99
follow commment
AngersZhuuuu Oct 17, 2019
7ac4d16
scalastyle
AngersZhuuuu Oct 17, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Next Next commit
fix show functions
  • Loading branch information
@AngersZhuuuu
AngersZhuuuu committed Oct 8, 2019
commit 911007a45f39fd3570613d73c94381eed79d3ac0
2 changes: 1 addition & 1 deletion sql/core/src/main/scala/org/apache/spark/sql/execution/command/functions.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -222,6 +222,6 @@ case class ShowFunctionsCommand(
case (f, "USER") if showUserFunctions => f.unquotedString
case (f, "SYSTEM") if showSystemFunctions => f.unquotedString
}
functionNames.sorted.map(Row(_))
(functionNames ++ Seq("!=", "<>", "between", "case")).sorted.map(Row(_))
}
}
3 changes: 2 additions & 1 deletion sql/core/src/test/scala/org/apache/spark/sql/SQLQuerySuite.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -80,7 +80,8 @@ class SQLQuerySuite extends QueryTest with SharedSparkSession {

createFunction(functions)

checkAnswer(sql("SHOW functions"), getFunctions("*"))
checkAnswer(sql("SHOW functions"), (getFunctions("*") ++
Seq(Row("!="), Row("<>"), Row("between"), Row("case"))))
Copy link
Contributor

@cloud-fan cloud-fan Oct 8, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: shall we put this code in getFunctions?

Copy link
Contributor Author

@AngersZhuuuu AngersZhuuuu Oct 8, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: shall we put this code in getFunctions?

Good ideal, I will try this.

assert(sql("SHOW functions").collect().size > 200)

Seq("^c*", "*e$", "log*", "*date*").foreach { pattern =>
Expand Down
4 changes: 2 additions & 2 deletions sql/core/src/test/scala/org/apache/spark/sql/execution/command/DDLSuite.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2065,12 +2065,12 @@ abstract class DDLSuite extends QueryTest with SQLTestUtils {
test("show functions") {
withUserDefinedFunction("add_one" -> true) {
val numFunctions = FunctionRegistry.functionSet.size.toLong
Copy link
Contributor

@cloud-fan cloud-fan Oct 8, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

shall we update it with val numFunctions = ... + 4?

assert(sql("show functions").count() === numFunctions)
assert(sql("show functions").count() === numFunctions + 4L)
assert(sql("show system functions").count() === numFunctions)
assert(sql("show all functions").count() === numFunctions)
assert(sql("show user functions").count() === 0L)
spark.udf.register("add_one", (x: Long) => x + 1)
assert(sql("show functions").count() === numFunctions + 1L)
assert(sql("show functions").count() === numFunctions + 5L)
assert(sql("show system functions").count() === numFunctions)
assert(sql("show all functions").count() === numFunctions + 1L)
assert(sql("show user functions").count() === 1L)
Expand Down
4 changes: 2 additions & 2 deletions sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveUDFSuite.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -563,8 +563,8 @@ class HiveUDFSuite extends QueryTest with TestHiveSingleton with SQLTestUtils {
checkAnswer(
sql("SELECT testUDFToListInt(s) FROM inputTable"),
Seq(Row(Seq(1, 2, 3))))
assert(sql("show functions").count() == numFunc + 1)
assert(spark.catalog.listFunctions().count() == numFunc + 1)
assert(sql("show functions").count() == numFunc + 5)
Copy link
Contributor

@cloud-fan cloud-fan Oct 17, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto

Copy link
Contributor Author

@AngersZhuuuu AngersZhuuuu Oct 18, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto

Could you help to trigger retest ?

assert(spark.catalog.listFunctions().count() == numFunc + 5)
}
}
}
Expand Down
5 changes: 5 additions & 0 deletions sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/SQLQuerySuite.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -192,6 +192,11 @@ class SQLQuerySuite extends QueryTest with SQLTestUtils with TestHiveSingleton {
allBuiltinFunctions.foreach { f =>
assert(allFunctions.contains(f))
}

Seq("!=", "<>", "between", "case").foreach { f =>
assert(allFunctions.contains(f))
}

withTempDatabase { db =>
def createFunction(names: Seq[String]): Unit = {
names.foreach { name =>
Expand Down