[BUG] Auto-normalize Delta table properties without the 'delta.' prefix #5985
Description
Bug
Which Delta project/connector is this regarding?
- Spark
- Standalone
- Flink
- Kernel
- Other (fill in here)
Describe the problem
Delta Lake table properties require a delta. prefix (e.g., delta.logRetentionDuration), but users often forget or don't know to include this prefix when creating or altering tables.
When users specified properties without the prefix (e.g., just logRetentionDuration), the system treats it as an arbitrary non-Delta property, silently ignoring the user's intended configuration
Steps to reproduce
Create a Delta table using a table property that does not include the delta. prefix
CREATE TABLE my_table (id INT, name STRING)
USING delta
TBLPROPERTIES ('logRetentionDuration' = 'interval 14 days')
Check the table properties
SHOW TBLPROPERTIES <table_name>;
Observed results
The property was stored as a generic/arbitrary property, not as a Delta configuration.
The Delta default of 30 days was still in use because the system didn't recognize logRetentionDuration without the delta. prefix.
Expected results
The log retention should be set to 14 days.
Further details
Environment information
- Delta Lake version:
- Spark version:
- Scala version:
Willingness to contribute
The Delta Lake Community encourages bug fix contributions. Would you or another member of your organization be willing to contribute a fix for this bug to the Delta Lake code base?
- Yes. I can contribute a fix for this bug independently.
- Yes. I would be willing to contribute a fix for this bug with guidance from the Delta Lake community.
- No. I cannot contribute a bug fix at this time.