-
Notifications
You must be signed in to change notification settings - Fork 29k
[SPARK-7562][SPARK-6444][SQL] Improve error reporting for expression data type mismatch #6405
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 1 commit
cb77e4f
7ae76b9
6491721
c71d02c
69ca3fe
1524ff6
e0a3628
654d46a
3affbd8
6eaadff
cffb67c
8883025
3bee157
0808fd2
39929d9
b917275
b5ff31b
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
- Loading branch information
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -86,12 +86,16 @@ abstract class Expression extends TreeNode[Expression] { | |
| case (i1, i2) => i1 == i2 | ||
| } | ||
| } | ||
|
|
||
| def typeMismatchErrorMessage: Option[String] = None | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. how about rename this and remove |
||
|
|
||
| def validInputTypes: Boolean = typeMismatchErrorMessage.isEmpty | ||
| } | ||
|
|
||
| abstract class BinaryExpression extends Expression with trees.BinaryNode[Expression] { | ||
| self: Product => | ||
|
|
||
| def symbol: String | ||
| def symbol: String = sys.error(s"BinaryExpressions must either override toString or symbol") | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Nit: please swap "either" and "override". |
||
|
|
||
| override def foldable: Boolean = left.foldable && right.foldable | ||
|
|
||
|
|
@@ -106,6 +110,10 @@ abstract class LeafExpression extends Expression with trees.LeafNode[Expression] | |
|
|
||
| abstract class UnaryExpression extends Expression with trees.UnaryNode[Expression] { | ||
| self: Product => | ||
|
|
||
| override def foldable: Boolean = child.foldable | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. this is not always correct, is it?
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. same for nulalble
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think we should just revert this change, and push it into the individual expressions or their base classes. Otherwise it is too error prone. If we add an expression that has different behavior, we will likely forget to change foldable/nullable.
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Should we also remove the |
||
|
|
||
| override def nullable: Boolean = child.nullable | ||
| } | ||
|
|
||
| // TODO Semantically we probably not need GroupExpression | ||
|
|
@@ -125,7 +133,9 @@ case class GroupExpression(children: Seq[Expression]) extends Expression { | |
| * so that the proper type conversions can be performed in the analyzer. | ||
| */ | ||
| trait ExpectsInputTypes { | ||
| self: Expression => | ||
|
|
||
| def expectedChildTypes: Seq[DataType] | ||
|
|
||
| override def validInputTypes: Boolean = children.map(_.dataType) == expectedChildTypes | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. what happened to error reporting if the type is invalid?
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We will always do type casting for
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Actually I don't like the handling of
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. That should be fine right? Also - add inline comment explaining this (i.e. things get casted). |
||
| } | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Since we separate type checking logic from
resolved, we need to ensure all children have valid input types before we do type coercion(think aboutAdd(Add(decimal1, decimal2), decimal3)). Here I just usetransformExpressionsUpas a quick fix(and only forDecimalPrecisionrule as it breaks a test...), should we skip not only!childrenResolvedbut also!children.forall(_.validInputTypes)during all type coercion, or still checking types inresolved? @rxin @marmbrus