Skip to content

Commit 58162b9

Browse files
committed
[SPARK-14387][SQL] Exceptions thrown when querying ORC tables
1 parent dc943a4 commit 58162b9

File tree

1 file changed

+8
-11
lines changed

1 file changed

+8
-11
lines changed

sql/hive/src/main/scala/org/apache/spark/sql/hive/orc/OrcFileFormat.scala

Lines changed: 8 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -105,17 +105,14 @@ private[sql] class OrcFileFormat
105105
dataSchema: StructType,
106106
physicalSchema: StructType,
107107
requiredSchema: StructType): StructType = {
108-
/**
109-
* requiredSchema names might not match with physical schema names.
110-
*
111-
* This is especially true when data is generated via Hive wherein
112-
* orc files would have column names as _col0, _col1 etc. This is
113-
* fixed in Hive 2.0, where in physical col names would match that
114-
* of metastore.
115-
*
116-
* To make it backward compatible, it is required to map physical
117-
* names to that of requiredSchema.
118-
*/
108+
109+
// requiredSchema names might not match with physical schema names.
110+
// This is especially true when data is generated via Hive wherein
111+
// orc files would have column names as _col0, _col1 etc. This is
112+
// fixed in Hive 2.0, where in physical col names would match that
113+
// of metastore. To make it backward compatible, it is required to
114+
// map physical names to that of requiredSchema.
115+
119116
// for requiredSchema, get the ordinal from dataSchema
120117
val ids = requiredSchema.map(a => dataSchema.fieldIndex(a.name): Integer).sorted
121118

0 commit comments

Comments
 (0)