Commit 286a376
[SPARK-37098][SQL] Alter table properties should invalidate cache
### What changes were proposed in this pull request?
Invalidate the table cache after alter table properties (set and unset).
### Why are the changes needed?
The table properties can change the behavior of wriing. e.g. the parquet table with `parquet.compression`.
If you execute the following SQL, we will get the file with snappy compression rather than zstd.
```
CREATE TABLE t (c int) STORED AS PARQUET;
// cache table metadata
SELECT * FROM t;
ALTER TABLE t SET TBLPROPERTIES('parquet.compression'='zstd');
INSERT INTO TABLE t values(1);
```
So we should invalidate the table cache after alter table properties.
### Does this PR introduce _any_ user-facing change?
yes, bug fix
### How was this patch tested?
Add test
Closes #34365 from ulysses-you/SPARK-37098.
Authored-by: ulysses-you <ulyssesyou18@gmail.com>
Signed-off-by: Kent Yao <yao@apache.org>
(cherry picked from commit 02d3b3b)
Signed-off-by: Kent Yao <yao@apache.org>1 parent c23698f commit 286a376
File tree
2 files changed
+26
-0
lines changed- sql
- core/src/main/scala/org/apache/spark/sql/execution/command
- hive/src/test/scala/org/apache/spark/sql/hive
2 files changed
+26
-0
lines changedLines changed: 2 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
277 | 277 | | |
278 | 278 | | |
279 | 279 | | |
| 280 | + | |
280 | 281 | | |
281 | 282 | | |
282 | 283 | | |
| |||
315 | 316 | | |
316 | 317 | | |
317 | 318 | | |
| 319 | + | |
318 | 320 | | |
319 | 321 | | |
320 | 322 | | |
| |||
Lines changed: 24 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
123 | 123 | | |
124 | 124 | | |
125 | 125 | | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
126 | 150 | | |
0 commit comments