Skip to content

[SPARK-19218][SQL] Fix SET command to show a result correctly and in a sorted order #16579

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
dongjoon-hyun wants to merge 9 commits into from
Closed

[SPARK-19218][SQL] Fix SET command to show a result correctly and in a sorted order #16579

Changes from 1 commit
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
c182c7d
[SPARK-19218][SQL] SET command should show a sorted result
dongjoon-hyun Jan 13, 2017
2ed6d87
Address comments.
dongjoon-hyun Jan 13, 2017
fa4914b
Revert to the approved commit and remove `set -v` test case to avoid …
dongjoon-hyun Jan 21, 2017
fb578d1
Add fix and recover `SET -v` test case.
dongjoon-hyun Jan 22, 2017
0214fad
Address comments
dongjoon-hyun Jan 22, 2017
528b0fd
Use Option.
dongjoon-hyun Jan 23, 2017
387ab59
Add `unregister` and use it.
dongjoon-hyun Jan 23, 2017
7061cd9
Remove redundant one.
dongjoon-hyun Jan 23, 2017
7879201
Add a missing clear() while splitting the testcases
dongjoon-hyun Jan 23, 2017
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Remove redundant one.
  • Loading branch information
@dongjoon-hyun
dongjoon-hyun committed Jan 23, 2017
commit 7061cd9ccd5684301efb2c6c6a8b05af36f65417
1 change: 0 additions & 1 deletion sql/core/src/test/scala/org/apache/spark/sql/SQLQuerySuite.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1003,7 +1003,6 @@ class SQLQuerySuite extends QueryTest with SharedSQLContext {
try {
val result = sql("SET -v").collect()
assert(result === result.sortBy(_.getString(0)))
Copy link
Member

@viirya viirya Jan 23, 2017 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh, i meant that you actually don't need a try {...} finally {...} here. you don't catch anything.

Copy link
Member Author

@dongjoon-hyun dongjoon-hyun Jan 23, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, I see what you meant. Actually, previously, SET -v raises exceptions, so this case use try and catch. But, as you mentioned, now it's not.

Copy link
Member Author

@dongjoon-hyun dongjoon-hyun Jan 23, 2017 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

However, IMO, it's needed to clean up spark.test if there occurs some regression for this case in the future. For the exceptions in that case, we needs that regression cause test cases failures, so catch is not used here.

Copy link
Member

@viirya viirya Jan 23, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

but you don't catch anything actually? so if any regression in the future, is it different with a try or not? you still see an exception.

Copy link
Member Author

@dongjoon-hyun dongjoon-hyun Jan 23, 2017 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, but we need to clean up spark.test in order not to interfere the other test cases here.

Copy link
Member

@viirya viirya Jan 23, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it is failed. isn't it??

Copy link
Member Author

@dongjoon-hyun dongjoon-hyun Jan 23, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:) The point is the other test cases are still running.

Copy link
Member Author

@dongjoon-hyun dongjoon-hyun Jan 23, 2017 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe, we are confused on terms.

  • You meant the other test statements.
  • I meant the other test cases

Copy link
Member

@viirya viirya Jan 23, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh, i meant the final Jenkins test result is failed. nvm, i think it is still useful so we can better infer which test causes the failure if we don't interfere other tests.

Copy link
Member Author

@dongjoon-hyun dongjoon-hyun Jan 23, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, I understand. Thanks. :)

spark.sessionState.conf.clear()
} finally {
Copy link
Member

@viirya viirya Jan 23, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: try ... finally seems redundant.

SQLConf.unregister(confEntry)
}
Expand Down