Skip to content

[SPARK-48658][SQL] Encode/Decode functions report coding errors instead of mojibake for unmappable characters #47017

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
yaooqinn wants to merge 12 commits into from
Closed

[SPARK-48658][SQL] Encode/Decode functions report coding errors instead of mojibake for unmappable characters #47017

Changes from 1 commit
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
6426fdd
[SPARK-48658][SQL] Encode/Decode functions report coding error instea…
yaooqinn Jun 19, 2024
aee78a5
[SPARK-48658][SQL] Encode/Decode functions report coding error instea…
yaooqinn Jun 19, 2024
afb2d08
[SPARK-48658][SQL] Encode/Decode functions report coding error instea…
yaooqinn Jun 19, 2024
f6dd4fa
fix ExplainSuite
yaooqinn Jun 19, 2024
851135c
fix golden file tests
yaooqinn Jun 19, 2024
d3473a4
fix golden file tests
yaooqinn Jun 19, 2024
3e90976
Encode RuntimeReplaceable with StaticInvoke
yaooqinn Jun 20, 2024
b0cf6ee
Decode RuntimeReplaceable with StaticInvoke
yaooqinn Jun 20, 2024
64f3c39
fix tests
yaooqinn Jun 20, 2024
9d2583c
Merge branch 'master' into SPARK-48658
yaooqinn Jun 20, 2024
8f5a236
fix
yaooqinn Jun 20, 2024
d7a4199
address comments
yaooqinn Jun 24, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
fix ExplainSuite
  • Loading branch information
@yaooqinn
yaooqinn committed Jun 19, 2024
commit f6dd4fa985085a080da8d70645ea89be92dfa363
8 changes: 4 additions & 4 deletions sql/core/src/test/scala/org/apache/spark/sql/ExplainSuite.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -193,8 +193,8 @@ class ExplainSuite extends ExplainSuiteHelper with DisableAdaptiveExecutionSuite
""".stripMargin)
checkKeywordsExistsInExplain(df2,
"Project [concat(cast(id#xL as string), cast((id#xL + 1) as string), " +
"cast(encode(cast((id#xL + 2) as string), utf-8, false) as string), " +
"cast(encode(cast((id#xL + 3) as string), utf-8, false) as string)) AS col#x]")
"cast(encode(cast((id#xL + 2) as string), utf-8, false, false) as string), " +
"cast(encode(cast((id#xL + 3) as string), utf-8, false, false) as string)) AS col#x]")

val df3 = sql(
"""
Expand All @@ -209,8 +209,8 @@ class ExplainSuite extends ExplainSuiteHelper with DisableAdaptiveExecutionSuite
""".stripMargin)
checkKeywordsExistsInExplain(df3,
"Project [concat(cast(id#xL as string), " +
"cast(encode(cast((id#xL + 2) as string), utf-8, false) as string), " +
"cast(encode(cast((id#xL + 3) as string), utf-8, false) as string)) AS col#x]")
"cast(encode(cast((id#xL + 2) as string), utf-8, false, false) as string), " +
"cast(encode(cast((id#xL + 3) as string), utf-8, false, false) as string)) AS col#x]")
}
}

Expand Down