Commit d9477dd
[SPARK-39376][SQL] Hide duplicated columns in star expansion of subquery alias from NATURAL/USING JOIN
### What changes were proposed in this pull request?
Follows up from #31666. This PR introduced a bug where the qualified star expansion of a subquery alias containing a NATURAL/USING output duplicated columns.
### Why are the changes needed?
Duplicated, hidden columns should not be output from a star expansion.
### Does this PR introduce _any_ user-facing change?
The query
```
val df1 = Seq((3, 8)).toDF("a", "b")
val df2 = Seq((8, 7)).toDF("b", "d")
val joinDF = df1.join(df2, "b")
joinDF.alias("r").select("r.*")
```
Now outputs a single column `b`, instead of two (duplicate) columns for `b`.
### How was this patch tested?
UTs
Closes #36763 from karenfeng/SPARK-39376.
Authored-by: Karen Feng <karen.feng@databricks.com>
Signed-off-by: Wenchen Fan <wenchen@databricks.com>1 parent 4a529a0 commit d9477dd
File tree
2 files changed
+24
-1
lines changed- sql
- catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical
- core/src/test/scala/org/apache/spark/sql
2 files changed
+24
-1
lines changedLines changed: 2 additions & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1303 | 1303 | | |
1304 | 1304 | | |
1305 | 1305 | | |
1306 | | - | |
| 1306 | + | |
| 1307 | + | |
1307 | 1308 | | |
1308 | 1309 | | |
1309 | 1310 | | |
| |||
Lines changed: 22 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
499 | 499 | | |
500 | 500 | | |
501 | 501 | | |
| 502 | + | |
| 503 | + | |
| 504 | + | |
| 505 | + | |
| 506 | + | |
| 507 | + | |
| 508 | + | |
| 509 | + | |
| 510 | + | |
| 511 | + | |
| 512 | + | |
| 513 | + | |
| 514 | + | |
| 515 | + | |
| 516 | + | |
| 517 | + | |
| 518 | + | |
| 519 | + | |
| 520 | + | |
| 521 | + | |
| 522 | + | |
| 523 | + | |
502 | 524 | | |
0 commit comments