Skip to content

[SPARK-9955][SQL] correct error message for aggregate #8203

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
cloud-fan wants to merge 2 commits into from
Closed

[SPARK-9955][SQL] correct error message for aggregate #8203

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
7593080
correct error message for aggregate
cloud-fan Aug 14, 2015
1c67ca7
move test
cloud-fan Aug 15, 2015
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -931,6 +931,7 @@ class Analyzer(
*/
object PullOutNondeterministic extends Rule[LogicalPlan] {
override def apply(plan: LogicalPlan): LogicalPlan = plan resolveOperators {
case p if !p.resolved => p // Skip unresolved nodes.
case p: Project => p
case f: Filter => f

Expand Down
18 changes: 12 additions & 6 deletions sql/core/src/test/scala/org/apache/spark/sql/SQLQuerySuite.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1618,12 +1618,18 @@ class SQLQuerySuite extends QueryTest with SharedSQLContext {
}

test("SPARK-9511: error with table starting with number") {
val df = sqlContext.sparkContext.parallelize(1 to 10).map(i => (i, i.toString))
.toDF("num", "str")
df.registerTempTable("1one")

checkAnswer(sql("select count(num) from 1one"), Row(10))
withTempTable("1one") {
sqlContext.sparkContext.parallelize(1 to 10).map(i => (i, i.toString))
.toDF("num", "str")
.registerTempTable("1one")
checkAnswer(sql("select count(num) from 1one"), Row(10))
}
}

sqlContext.dropTempTable("1one")
test("SPARK-9955: correct error message for aggregate") {
val e = intercept[AnalysisException] {
sql("select key, max(value) from testData where bad_column > 1 group by key")
Copy link
Contributor

@marmbrus marmbrus Aug 14, 2015

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you please move this to AnalysisErrorSuite?

Copy link
Contributor Author

@cloud-fan cloud-fan Aug 15, 2015

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The UnresolvedAlias is only introduced when parsing SQL or DataFrame, so if we move this test into AnalysisErrorSuite, we need to add the UnresolvedAliases manually, is that OK?

Copy link
Contributor

@marmbrus marmbrus Aug 15, 2015

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or construct the plan by hand? Or we could maybe start a dataframe error
suite.
On Aug 14, 2015 8:26 PM, "Wenchen Fan" [email protected] wrote:

In sql/core/src/test/scala/org/apache/spark/sql/SQLQuerySuite.scala
#8203 (comment):

  • sqlContext.dropTempTable("1one")
  • test("SPARK-9955: correct error message for aggregate") {
  • val e = intercept[AnalysisException] {
  •  sql("select key, max(value) from testData where bad_column > 1 group by key")

The UnresolvedAlias is only introduced when parsing SQL or DataFrame, so
if we move this test into AnalysisErrorSuite, we need to add the
UnresolvedAliases manually, is that OK?

—
Reply to this email directly or view it on GitHub
https://github.com/apache/spark/pull/8203/files#r37132757.

}
assert(e.message.contains("cannot resolve 'bad_column'"))
}
}