Skip to content

Conversation

@MaxGekk
Copy link
Member

@MaxGekk MaxGekk commented Oct 31, 2022

What changes were proposed in this pull request?

In the PR, I propose to assign the proper name UNSUPPORTED_TYPED_LITERAL to the legacy error class _LEGACY_ERROR_TEMP_0021 , and modify test suite to use checkError() which checks the error class name, context and etc.

Why are the changes needed?

Proper name improves user experience w/ Spark SQL.

Does this PR introduce any user-facing change?

Yes, the PR changes an user-facing error message.

How was this patch tested?

By running the modified test suites:

$ PYSPARK_PYTHON=python3 build/sbt "sql/testOnly org.apache.spark.sql.SQLQueryTestSuite"
$ build/sbt "test:testOnly *ExpressionParserSuite"
$ build/sbt "test:testOnly *AnalysisSuite"

@MaxGekk MaxGekk changed the title [WIP][SQL] Rename the error class _LEGACY_ERROR_TEMP_0021 to UNSUPPORTED_TYPED_LITERAL [WIP][SPARK-40975][SQL] Rename the error class _LEGACY_ERROR_TEMP_0021 to UNSUPPORTED_TYPED_LITERAL Oct 31, 2022
@MaxGekk MaxGekk changed the title [WIP][SPARK-40975][SQL] Rename the error class _LEGACY_ERROR_TEMP_0021 to UNSUPPORTED_TYPED_LITERAL [SPARK-40975][SQL] Rename the error class _LEGACY_ERROR_TEMP_0021 to UNSUPPORTED_TYPED_LITERAL Oct 31, 2022
@MaxGekk MaxGekk marked this pull request as ready for review October 31, 2022 10:57
@MaxGekk MaxGekk requested a review from cloud-fan October 31, 2022 12:41
@MaxGekk
Copy link
Member Author

MaxGekk commented Oct 31, 2022

The failure is not related to my changes:

ERROR: test_await_termination_or_timeout (pyspark.streaming.tests.test_context.StreamingContextTests)

The same finished successfully on my laptop.

@MaxGekk
Copy link
Member Author

MaxGekk commented Oct 31, 2022

@cloud-fan @itholic @srielau @LuciferYang @panbingkun Please, review this PR.

throw QueryParsingErrors.literalValueTypeUnsupportedError(
unsupportedType = other,
supportedTypes =
Seq("DATE", "TIMESTAMP_NTZ", "TIMESTAMP_LTZ", "TIMESTAMP", "INTERVAL", "X"),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's X? Hexadecimal?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe a lowercase hexadecimal would be less confusing?

Copy link
Member Author

@MaxGekk MaxGekk Oct 31, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here, I pointed out the exact keyword that can be used in the typed literals, see

The syntax is:

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could give a hint to user like X hexadecimal

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you mean "X" -> "X hexadecimal" ?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, but I would like it as is. Inconsistency confuses always. If we point out supported "types" in typed literals, we should show supported words only.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree with you, "X hexadecimal" looks a little strange

Copy link
Contributor

@LuciferYang LuciferYang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1, fine to me

Copy link
Contributor

@itholic itholic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM when the test passing

"errorClass" : "UNSUPPORTED_TYPED_LITERAL",
"messageParameters" : {
"valueType" : "GEO"
"supportedTypes" : "\"DATE\", \"TIMESTAMP_NTZ\", \"TIMESTAMP_LTZ\", \"TIMESTAMP\", \"INTERVAL\", \"X\"",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

isn't it a constant list? does it need to be a parameter?

Copy link
Member Author

@MaxGekk MaxGekk Nov 1, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It depends on implementation, and we might support more in the future. Having the list in the source code can give the following benefits:

  1. The list of parameters might be formatted in different ways ( as a foldable or drop-down list) by frontend tools.
  2. The error classes can be re-used from another places with different list.
  3. More likely, devs will not forget to update the list in source code.
  4. If the list will be in the JSON file, tech editors might forget to edit it or accidentally modify it.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it should be a parameter, unless UNSUPPORTED_TYPED_LITERAL will only be used in this scenario

@MaxGekk
Copy link
Member Author

MaxGekk commented Nov 1, 2022

Merging to master. Thank you, @cloud-fan @LuciferYang @itholic @srielau for review.

@MaxGekk MaxGekk closed this in 9bbc9ee Nov 1, 2022
SandishKumarHN pushed a commit to SandishKumarHN/spark that referenced this pull request Dec 12, 2022
…o `UNSUPPORTED_TYPED_LITERAL`

### What changes were proposed in this pull request?
In the PR, I propose to assign the proper name `UNSUPPORTED_TYPED_LITERAL ` to the legacy error class `_LEGACY_ERROR_TEMP_0021 `, and modify test suite to use `checkError()` which checks the error class name, context and etc.

### Why are the changes needed?
Proper name improves user experience w/ Spark SQL.

### Does this PR introduce _any_ user-facing change?
Yes, the PR changes an user-facing error message.

### How was this patch tested?
By running the modified test suites:
```
$ PYSPARK_PYTHON=python3 build/sbt "sql/testOnly org.apache.spark.sql.SQLQueryTestSuite"
$ build/sbt "test:testOnly *ExpressionParserSuite"
$ build/sbt "test:testOnly *AnalysisSuite"
```

Closes apache#38448 from MaxGekk/error-class-typed-literal.

Authored-by: Max Gekk <[email protected]>
Signed-off-by: Max Gekk <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants