Skip to content

[SPARK-16948][SQL] Use metastore schema instead of inferring schema for ORC in HiveMetastoreCatalog #14537

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
rajeshbalamohan wants to merge 20 commits into from
Closed

[SPARK-16948][SQL] Use metastore schema instead of inferring schema for ORC in HiveMetastoreCatalog #14537

Changes from 1 commit
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
5721b88
[SPARK-16948][SQL] Querying empty partitioned orc tables throws excep…
rbalamohan Aug 8, 2016
4ae92d8
[SPARK-16948][SQL] Querying empty partitioned orc tables throws excep…
rbalamohan Aug 9, 2016
75bca1b
[SPARK-16948][SQL] Querying empty partitioned orc tables throws excep…
rbalamohan Aug 9, 2016
97d21f6
Merge branch 'master' of https://github.com/apache/spark into SPARK-1…
rbalamohan Aug 22, 2016
4004c0a
[SPARK-16948][SQL] Querying empty partitioned orc tables throws excep…
rbalamohan Aug 22, 2016
9a8838a
[SPARK-16948][SQL] Querying empty partitioned orc tables throws excep…
rbalamohan Aug 23, 2016
eb8a955
[SPARK-16948][SQL] Querying empty partitioned orc tables throws excep…
rbalamohan Aug 23, 2016
70cf84d
[SPARK-16948][SQL] Querying empty partitioned orc tables throws excep…
rbalamohan Aug 24, 2016
c9d677b
[SPARK-16948][SQL] Querying empty partitioned orc tables throws excep…
rbalamohan Aug 24, 2016
7385a06
Revert "[SPARK-16948][SQL] Querying empty partitioned orc tables thro…
rbalamohan Aug 24, 2016
1746853
Revert "Revert "[SPARK-16948][SQL] Querying empty partitioned orc tab…
rbalamohan Aug 24, 2016
5f7e5ad
[SPARK-16948][SQL] Querying empty partitioned orc tables throws excep…
rbalamohan Aug 24, 2016
e8e2d70
[SPARK-16948][SQL] Querying empty partitioned orc tables throws excep…
rbalamohan Aug 25, 2016
0f901aa
[SPARK-16948][SQL] Querying empty partitioned orc tables throws excep…
rbalamohan Aug 25, 2016
0772a96
[SPARK-16948][SQL] Querying empty partitioned orc tables throws excep…
rbalamohan Aug 25, 2016
6ff7e5d
[SPARK-16948][SQL] Querying empty partitioned orc tables throws excep…
rbalamohan Aug 25, 2016
fc14e2d
[SPARK-16948][SQL] Querying empty partitioned orc tables throws excep…
rbalamohan Aug 25, 2016
9ecb2ed
[SPARK-16948][SQL] Querying empty partitioned orc tables throws excep…
rbalamohan Aug 26, 2016
fa71370
[SPARK-16948][SQL] Querying empty partitioned orc tables throws excep…
rbalamohan Sep 22, 2016
e39715e
[SPARK-16948][SQL] Querying empty partitioned orc tables throws excep…
rbalamohan Sep 22, 2016
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
[SPARK-16948][SQL] Querying empty partitioned orc tables throws excep… 
…tion
  • Loading branch information
@rbalamohan
rbalamohan committed Aug 23, 2016
commit eb8a955e9fd39b2c8b9b88cbc1f01b1937af1189
2 changes: 1 addition & 1 deletion sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveMetastoreCatalog.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -249,7 +249,7 @@ private[hive] class HiveMetastoreCatalog(sparkSession: SparkSession) extends Log
case "orc" =>
inferredSchema.getOrElse(metastoreSchema)
case _ =>
inferredSchema.get
throw new RuntimeException(s"Cannot convert a $fileType to a HadoopFsRelation")
Copy link
Member

@gatorsmile gatorsmile Aug 25, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Spark SQL users do not know what is a HadoopFsRelation.

to a data source table?

}

val relation = HadoopFsRelation(
Expand Down