Skip to content

[SPARK-17183][SPARK-17983][SPARK-18101][SQL] put hive serde table schema to table properties like data source table #14750

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
cloud-fan wants to merge 8 commits into from
Closed

[SPARK-17183][SPARK-17983][SPARK-18101][SQL] put hive serde table schema to table properties like data source table #14750

Changes from 1 commit
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
17321a3
put hive serde table schema to table properties like data source table
cloud-fan Nov 3, 2016
7b6fb13
follow up
cloud-fan Nov 3, 2016
886a5af
minor
cloud-fan Nov 3, 2016
460302c
Merge remote-tracking branch 'origin/master' into minor1
cloud-fan Nov 4, 2016
afa5d85
address comments
cloud-fan Nov 4, 2016
b02cf0d
enable a test
cloud-fan Nov 4, 2016
c057d0c
address comments
cloud-fan Nov 5, 2016
3bd9362
minor
cloud-fan Nov 5, 2016
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
minor
  • Loading branch information
@cloud-fan
cloud-fan committed Nov 5, 2016
commit 3bd9362062b41ad64146cd24936501f4dfd65b68
4 changes: 2 additions & 2 deletions sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -651,8 +651,8 @@ private[spark] class HiveExternalCatalog(conf: SparkConf, hadoopConf: Configurat
// from hive.
logWarning(s"The table schema given by Hive metastore(${table.schema.simpleString}) is " +
"different from the schema when this table was created by Spark SQL" +
s"(${schemaFromTableProps.simpleString}). We have to trust the table schema from Hive " +
"metastore which is not case preserving.")
s"(${schemaFromTableProps.simpleString}). We have to fall back to the table schema from " +
"Hive metastore which is not case preserving.")
hiveTable
Copy link
Contributor

@ericl ericl Nov 4, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are there cases where this could be issue? Should we log a warning here?

Copy link
Contributor Author

@cloud-fan cloud-fan Nov 5, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good idea!

}
}
Expand Down