Skip to content

[SPARK-26493][SQL] Allow multiple spark.sql.extensions #23398

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
jamisonbennett wants to merge 7 commits into from
+198 −25
Closed

[SPARK-26493][SQL] Allow multiple spark.sql.extensions #23398

Changes from 1 commit
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
cef8eb0
[SPARK-26493][SQL] Allow multiple spark.sql.extensions
jamisonbennett Dec 28, 2018
689a4d2
Address comments from dongjoon-hyun, hvanhovell, beliefer, gatorsmile
jamisonbennett Dec 29, 2018
d89cfd9
Address comment from hvanhovell
jamisonbennett Dec 31, 2018
fb4ad34
Address comments from HyukjinKwon
jamisonbennett Jan 5, 2019
65a5f3f
Address comments from HyukjinKwon, vanzin
jamisonbennett Jan 9, 2019
9c0181d
Fix a typo
jamisonbennett Jan 9, 2019
deaf73e
Address comments from vanzin
jamisonbennett Jan 9, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Address comment from hvanhovell 
This addresses the comments for #23398
  • Loading branch information
@jamisonbennett
jamisonbennett committed Dec 31, 2018
commit d89cfd9956f7205ed2b2d1b552cca43b2471bfcb
7 changes: 5 additions & 2 deletions sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/FunctionRegistry.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -105,8 +105,11 @@ class SimpleFunctionRegistry extends FunctionRegistry with Logging {
info: ExpressionInfo,
builder: FunctionBuilder): Unit = synchronized {
val normalizedName = normalizeFuncName(name)
if (functionBuilders.put(normalizedName, (info, builder)).isDefined) {
logWarning(s"The function $normalizedName replaced a previously registered function.")
val newFunction = (info, builder)
functionBuilders.put(normalizedName, newFunction) match {
case Some(previousFunction) if previousFunction != newFunction =>
logWarning(s"The function $normalizedName replaced a previously registered function.")
case _ => Unit
Copy link
Member

@HyukjinKwon HyukjinKwon Jan 9, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is returning Unit type object. This can be just removed.

Copy link
Contributor Author

@jamisonbennett jamisonbennett Jan 9, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I made a change here which I think was what you were recommending. I didn't want to remove the case _ => otherwise I think it may result in a non-exhaustive match exception. If you wanted me to remove the entire match statement, just let me know.

}
}

Expand Down