Skip to content

[SPARK-17154][SQL] Wrong result can be returned or AnalysisException can be thrown after self-join or similar operations #14719

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
sarutak wants to merge 18 commits into from
Closed

[SPARK-17154][SQL] Wrong result can be returned or AnalysisException can be thrown after self-join or similar operations #14719

Changes from 1 commit
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
05872b7
Fixed the issue that self-join or similar join patterns can cause wro…
sarutak Aug 19, 2016
91cb915
Fix ScriptTransformationSuite
sarutak Aug 20, 2016
dd0ddbc
Fixed further more test cases
sarutak Aug 20, 2016
74eb4aa
Fixed analysis error in sorting
sarutak Aug 20, 2016
48a0775
Refectored
sarutak Aug 20, 2016
9ddc9d8
Fixed flatMapGroupsInR to ensure expressions for Deserializer is reso…
sarutak Aug 20, 2016
148b6d5
Fixed style
sarutak Aug 21, 2016
021977f
Implemented another idea
sarutak Aug 27, 2016
b09c0d7
Fixed conflict
sarutak Aug 27, 2016
ccf71fc
Modified error message
sarutak Aug 28, 2016
d458b79
Merge branch 'master' of git://git.apache.org/spark into SPARK-17154
sarutak Sep 28, 2016
437ac99
Reverted the previous change which prohibitted self-join
sarutak Sep 28, 2016
9ad2c85
Merge branch 'master' of git://git.apache.org/spark into SPARK-17154
sarutak Oct 11, 2016
929f2a8
Resolved conflict
sarutak Oct 28, 2016
15bf529
Fixed style
sarutak Oct 28, 2016
e91a24e
Merge branch 'master' of git://git.apache.org/spark into SPARK-17154
sarutak Nov 17, 2016
5d1ff3e
Fix compile error
sarutak Nov 17, 2016
a3f32c4
Fixed conflicts
sarutak Jan 5, 2017
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Fix ScriptTransformationSuite
  • Loading branch information
@sarutak
sarutak committed Aug 20, 2016
commit 91cb915b4e6c3c4d24fab3f1e772e7e361d4c088
10 changes: 5 additions & 5 deletions sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/ScriptTransformationSuite.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@ class ScriptTransformationSuite extends SparkPlanTest with TestHiveSingleton {
checkAnswer(
rowsDf,
(child: SparkPlan) => new ScriptTransformation(
input = Seq(rowsDf.col("a").expr),
input = Seq(rowsDf.colInternal("a").expr),
script = "cat",
output = Seq(AttributeReference("a", StringType)()),
child = child,
Expand All @@ -67,7 +67,7 @@ class ScriptTransformationSuite extends SparkPlanTest with TestHiveSingleton {
checkAnswer(
rowsDf,
(child: SparkPlan) => new ScriptTransformation(
input = Seq(rowsDf.col("a").expr),
input = Seq(rowsDf.colInternal("a").expr),
script = "cat",
output = Seq(AttributeReference("a", StringType)()),
child = child,
Expand All @@ -82,7 +82,7 @@ class ScriptTransformationSuite extends SparkPlanTest with TestHiveSingleton {
checkAnswer(
rowsDf,
(child: SparkPlan) => new ScriptTransformation(
input = Seq(rowsDf.col("a").expr),
input = Seq(rowsDf.colInternal("a").expr),
script = "cat",
output = Seq(AttributeReference("a", StringType)()),
child = ExceptionInjectingOperator(child),
Expand All @@ -99,7 +99,7 @@ class ScriptTransformationSuite extends SparkPlanTest with TestHiveSingleton {
checkAnswer(
rowsDf,
(child: SparkPlan) => new ScriptTransformation(
input = Seq(rowsDf.col("a").expr),
input = Seq(rowsDf.colInternal("a").expr),
script = "cat",
output = Seq(AttributeReference("a", StringType)()),
child = ExceptionInjectingOperator(child),
Expand All @@ -116,7 +116,7 @@ class ScriptTransformationSuite extends SparkPlanTest with TestHiveSingleton {
val e = intercept[SparkException] {
val plan =
new ScriptTransformation(
input = Seq(rowsDf.col("a").expr),
input = Seq(rowsDf.colInternal("a").expr),
script = "some_non_existent_command",
output = Seq(AttributeReference("a", StringType)()),
child = rowsDf.queryExecution.sparkPlan,
Expand Down