Skip to content

[SPARK-49666][SQL] Add feature flag for trim collation feature #48222

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
jovanpavl-db wants to merge 19 commits into from
Closed

[SPARK-49666][SQL] Add feature flag for trim collation feature #48222

Changes from 1 commit
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
b5a37e7
introduce space trimming flag in collation factory.
jovanpavl-db Sep 20, 2024
954c6c1
refactor CollationFactory.
jovanpavl-db Sep 22, 2024
803d187
fix collation name to id.
jovanpavl-db Sep 22, 2024
a880e02
minor fix.
jovanpavl-db Sep 23, 2024
01628d6
fix default space trimming.
jovanpavl-db Sep 23, 2024
5192476
feature flag implementation
jovanpavl-db Sep 23, 2024
a3c5e8c
Trim collation banned with collate builder
jovanpavl-db Sep 23, 2024
e65232e
implement collation trim block without collate builder call.
jovanpavl-db Sep 23, 2024
9635ab7
add test for set collation.
jovanpavl-db Sep 23, 2024
87cc344
add tests for recognition of space trimming.
jovanpavl-db Sep 24, 2024
587cd2b
minor fix
jovanpavl-db Sep 24, 2024
27a3058
address comments
jovanpavl-db Sep 24, 2024
58aab95
simpfly switch statement.
jovanpavl-db Sep 24, 2024
16c1674
fix test, address rest of the comments.
jovanpavl-db Sep 24, 2024
bc99ecb
nit fixes.
jovanpavl-db Sep 25, 2024
5328ab7
fix scala style.
jovanpavl-db Sep 25, 2024
156728c
fix java style.
jovanpavl-db Sep 25, 2024
9a1bc8b
addressing comments.
jovanpavl-db Sep 25, 2024
d4fb363
mark feature flag with internal.
jovanpavl-db Sep 26, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
add test for set collation.
  • Loading branch information
@jovanpavl-db
jovanpavl-db committed Sep 23, 2024
commit 9635ab7a5dcee110d8c22c6c6d94fedeb01ce45b
7 changes: 7 additions & 0 deletions sql/core/src/test/scala/org/apache/spark/sql/internal/SQLConfSuite.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -517,6 +517,13 @@ class SQLConfSuite extends QueryTest with SharedSparkSession {
"confName" -> "spark.sql.session.collation.default",
"proposals" -> "UNICODE"
))

withSQLConf(SQLConf.TRIM_COLLATION_ENABLED.key -> "false") {
checkError(
exception = intercept[AnalysisException](sql(s"SET COLLATION UNICODE_CI_TRIM")),
condition = "UNSUPPORTED_FEATURE.TRIM_COLLATION"
)
}
}

test("SPARK-43028: config not found error") {
Expand Down