Skip to content

[SPARK-43914][SQL] Assign names to the error class _LEGACY_ERROR_TEMP_[2433-2437] #41476

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
beliefer wants to merge 8 commits into from
Closed

[SPARK-43914][SQL] Assign names to the error class _LEGACY_ERROR_TEMP_[2433-2437] #41476

Changes from 1 commit
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
f6e8809
[SPARK-43914][SQL] Assign names to the error class _LEGACY_ERROR_TEMP…
beliefer Jun 6, 2023
e17cc59
Update code
beliefer Jun 6, 2023
3668c24
Update code
beliefer Jun 14, 2023
15987f9
Update code
beliefer Jun 18, 2023
169dfa5
Update code
beliefer Jun 19, 2023
d549eee
Update code
beliefer Jun 20, 2023
a563972
Update code
beliefer Jun 20, 2023
bc9b11f
Fix conflicts
beliefer Jun 21, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Update code
  • Loading branch information
@beliefer
beliefer committed Jun 23, 2023
commit e17cc594d02129c8348e378e6c1d621a2049181c
68 changes: 65 additions & 3 deletions sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/analysis/AnalysisErrorSuite.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ import org.apache.spark.sql.catalyst.dsl.plans._
import org.apache.spark.sql.catalyst.expressions._
import org.apache.spark.sql.catalyst.expressions.aggregate.{Count, Max}
import org.apache.spark.sql.catalyst.parser.CatalystSqlParser
import org.apache.spark.sql.catalyst.plans.{Cross, LeftOuter, RightOuter}
import org.apache.spark.sql.catalyst.plans.{AsOfJoinDirection, Cross, Inner, LeftOuter, RightOuter}
import org.apache.spark.sql.catalyst.plans.logical._
import org.apache.spark.sql.catalyst.util.{ArrayBasedMapData, GenericArrayData, MapData}
import org.apache.spark.sql.internal.SQLConf
Expand Down Expand Up @@ -784,8 +784,70 @@ class AnalysisErrorSuite extends AnalysisTest {
val error = intercept[AnalysisException] {
SimpleAnalyzer.checkAnalysis(join)
}
assert(error.message.contains("Failure when resolving conflicting references in Join"))
assert(error.message.contains("Conflicting attributes"))
checkError(
exception = error,
errorClass = "RESOLVED_PLAN_HAVE_CONFLICTING_ATTRS",
parameters = Map(
"nodeName" -> "Join",
"plan" -> "'Join Cross\n:- LocalRelation <empty>, [a#0]\n+- LocalRelation <empty>, [a#0]\n",
"conflictingAttributes" -> "\"a\""
)
)
}

test("error test for self-intersect") {
val intersect = Intersect(testRelation, testRelation, true)
val error = intercept[AnalysisException] {
SimpleAnalyzer.checkAnalysis(intersect)
Copy link
Contributor Author

@beliefer beliefer Jun 6, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@MaxGekk I created test cases use SimpleAnalyzer.checkAnalysis directly.

Copy link
Member

@MaxGekk MaxGekk Jun 18, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see but if you cannot trigger the error from user space, we should convert the error to an internal one.

Copy link
Member

@MaxGekk MaxGekk Jun 18, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@beliefer Could you try to trigger it by from SQL.

Copy link
Contributor Author

@beliefer beliefer Jun 18, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@beliefer Could you try to trigger it by from SQL.

We can't trigger it by SQL. Let me convert it to internal error.

Copy link
Member

@MaxGekk MaxGekk Jun 19, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cc @cloud-fan Just to double check that the errors are internal.

}
checkError(
exception = error,
errorClass = "RESOLVED_PLAN_HAVE_CONFLICTING_ATTRS",
parameters = Map(
"nodeName" -> "Intersect All",
"plan" ->
"'Intersect All true\n:- LocalRelation <empty>, [a#0]\n+- LocalRelation <empty>, [a#0]\n",
"conflictingAttributes" -> "\"a\""
)
)
}

test("error test for self-except") {
val except = Except(testRelation, testRelation, true)
val error = intercept[AnalysisException] {
SimpleAnalyzer.checkAnalysis(except)
}
checkError(
exception = error,
errorClass = "RESOLVED_PLAN_HAVE_CONFLICTING_ATTRS",
parameters = Map(
"nodeName" -> "Except All",
"plan" ->
"'Except All true\n:- LocalRelation <empty>, [a#0]\n+- LocalRelation <empty>, [a#0]\n",
"conflictingAttributes" -> "\"a\""
)
)
}

test("error test for self-asOfJoin") {
val asOfJoin =
AsOfJoin(testRelation, testRelation, testRelation.output(0), testRelation.output(0),
None, Inner, tolerance = None, allowExactMatches = true,
direction = AsOfJoinDirection("backward"))
val error = intercept[AnalysisException] {
SimpleAnalyzer.checkAnalysis(asOfJoin)
}
val expectedPlan = "'AsOfJoin (a#0 >= a#0), " +
"Inner\n:- LocalRelation <empty>, [a#0]\n+- LocalRelation <empty>, [a#0]\n"
checkError(
exception = error,
errorClass = "RESOLVED_PLAN_HAVE_CONFLICTING_ATTRS",
parameters = Map(
"nodeName" -> "AsOfJoin",
"plan" -> expectedPlan,
"conflictingAttributes" -> "\"a\""
)
)
}

test("check grouping expression data types") {
Expand Down