Skip to content

Conversation

@javierivanov
Copy link
Contributor

What changes were proposed in this pull request?

This PR adds a formatted print method like it is provided in the showString implementation from Dataset while using the spark-sql client.

Actual Spark-sql client

spark-sql> select * from table1;
5       name3   add3
5       name1   add1
5       name2   add2

Spark Dataset show

scala> sql("select * from table1").show()
+---+-----+-------+
| id| name|address|
+---+-----+-------+
|  5|name3|   add3|
|  5|name1|   add1|
|  5|name2|   add2|
+---+-----+-------+

Why are the changes needed?

To have a consistent output between Dataset show and spark-sql interfaces.

Does this PR introduce any user-facing change?

No

How was this patch tested?

org.apache.spark.sql.hive.thriftserver.CliSuite.scala was adapted for the new fortmat. Code compiles and pass tests.

@AmplabJenkins
Copy link

Can one of the admins verify this patch?

Copy link
Member

@dongjoon-hyun dongjoon-hyun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for making a PR, @javierivanov .
GitHub Action seems to complain about the Scala coding style.
Could you run dev/scalastyle and fix all of them?

@dongjoon-hyun dongjoon-hyun changed the title [Spark-29685][SQL] Adds format to the spark-sql output like its implemented by Dataset showString [SPARK-29685][SQL] Adds format to the spark-sql output like its implemented by Dataset showString Nov 28, 2019
@javierivanov
Copy link
Contributor Author

Thanks for the quick response @dongjoon-hyun. All good now.

"""select 'Test3', "\';";""" -> "Test3\t';",
"""select 'Test1', "^;^";""" -> "Test1|^;^",
"""select 'Test2', "\";";""" -> "Test2| \";",
"""select 'Test3', "\';";""" -> "Test3| ';",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the output was matched to Hive's intentionally.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you mean the extra space? the minimun col width is 3

@wangyum
Copy link
Member

wangyum commented Nov 30, 2019

Hi @javierivanov Are you interested in implementing a new Spark CLI similar to spark-sql, but does not include any Hive dependency.

@javierivanov
Copy link
Contributor Author

Yeah, sure. The idea is to have a pure Scala implementation ?

@javierivanov
Copy link
Contributor Author

@wangyum do you mean a different module or related to this PR #26340?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants