Commit 3d7a20a
[SPARK-48709][SQL][3.5] Fix varchar type resolution mismatch for DataSourceV2 CTAS
Backport of #47082.
### What changes were proposed in this pull request?
This PR fixes varchar type resolution mismatch for DataSourceV2 CTAS. For example:
```sql
set spark.sql.storeAssignmentPolicy=LEGACY;
CREATE TABLE testcat.ns.t1 (d1 string, d2 varchar(200)) USING parquet;
CREATE TABLE testcat.ns.t2 USING foo as select * from testcat.ns.t1
```
Error message:
```
org.apache.spark.sql.AnalysisException: LEGACY store assignment policy is disallowed in Spark data source V2. Please set the configuration spark.sql.storeAssignmentPolicy to other values.
```
### Why are the changes needed?
Avoid query failures.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Unit test.
### Was this patch authored or co-authored using generative AI tooling?
No.
Closes #47103 from wangyum/SPARK-48709-branch-3.5.
Authored-by: Yuming Wang <[email protected]>
Signed-off-by: Kent Yao <[email protected]>1 parent b28ddb1 commit 3d7a20a
File tree
2 files changed
+12
-1
lines changed- sql
- catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical
- core/src/test/scala/org/apache/spark/sql/connector
2 files changed
+12
-1
lines changedLines changed: 2 additions & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
62 | 62 | | |
63 | 63 | | |
64 | 64 | | |
| 65 | + | |
65 | 66 | | |
66 | 67 | | |
67 | 68 | | |
68 | | - | |
| 69 | + | |
69 | 70 | | |
70 | 71 | | |
71 | 72 | | |
| |||
Lines changed: 10 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1736 | 1736 | | |
1737 | 1737 | | |
1738 | 1738 | | |
| 1739 | + | |
| 1740 | + | |
| 1741 | + | |
| 1742 | + | |
| 1743 | + | |
| 1744 | + | |
| 1745 | + | |
| 1746 | + | |
| 1747 | + | |
| 1748 | + | |
1739 | 1749 | | |
1740 | 1750 | | |
1741 | 1751 | | |
| |||
0 commit comments