Skip to content

[SPARK-48441][SQL] Fix StringTrim behaviour for non-UTF8_BINARY collations #46762

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
uros-db wants to merge 21 commits into from
Closed

[SPARK-48441][SQL] Fix StringTrim behaviour for non-UTF8_BINARY collations #46762

Changes from 1 commit
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
0209fc4
Initial commit
uros-db May 27, 2024
1f6881d
Small fixes
uros-db May 28, 2024
204d26e
Merge branch 'apache:master' into alter-trim
uros-db May 28, 2024
5039ddf
Fix trim logic
uros-db May 30, 2024
361c7b1
Lint fixes
uros-db May 30, 2024
aaa46ff
Merge branch 'master' into alter-trim
uros-db Jun 10, 2024
1af57bf
Update CollationSupportSuite.java
uros-db Jun 10, 2024
544548c
Merge branch 'master' into alter-trim
uros-db Jul 4, 2024
e559e83
Merge branch 'apache:master' into alter-trim
uros-db Jul 5, 2024
292acf8
Fix test
uros-db Jul 7, 2024
097a846
Refactor trim
uros-db Jul 8, 2024
176b148
Add tests
uros-db Jul 8, 2024
9192b55
Merge branch 'apache:master' into alter-trim
uros-db Jul 8, 2024
598f522
Remove unused code
uros-db Jul 8, 2024
12cc084
Fix java lint
uros-db Jul 8, 2024
469c325
Fix
uros-db Jul 9, 2024
c2133d7
Merge branch 'apache:master' into alter-trim
uros-db Jul 9, 2024
06575fc
Fix comments
uros-db Jul 12, 2024
4cf90d7
Merge branch 'master' into alter-trim
uros-db Jul 12, 2024
8c5787d
Fixes
uros-db Jul 14, 2024
d15d92a
Fix tests
uros-db Jul 14, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Remove unused code
  • Loading branch information
@uros-db
uros-db committed Jul 8, 2024
commit 598f52216f8ca43dbb2a3c75f40cfc3d3aba4f70
12 changes: 0 additions & 12 deletions common/unsafe/src/main/java/org/apache/spark/sql/catalyst/util/CollationFactory.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -813,18 +813,6 @@ public static String[] getICULocaleNames() {
return Collation.CollationSpecICU.ICULocaleNames;
}

public static String getCollationKey(String input, int collationId) {
Collation collation = fetchCollation(collationId);
if (collation.supportsBinaryEquality) {
return input;
} else if (collation.supportsLowercaseEquality) {
return input.toLowerCase();
} else {
CollationKey collationKey = collation.collator.getCollationKey(input);
return Arrays.toString(collationKey.toByteArray());
}
}

public static UTF8String getCollationKey(UTF8String input, int collationId) {
Collation collation = fetchCollation(collationId);
if (collation.supportsBinaryEquality) {
Expand Down