Skip to content

SPARK-1294 Fix resolution of uppercase field names using a HiveContext. #202

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
marmbrus wants to merge 3 commits into from
Closed

SPARK-1294 Fix resolution of uppercase field names using a HiveContext. #202

Changes from 1 commit
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
9d99cb6
Attributes should be compared using exprId, not TreeNode.id.
marmbrus Mar 22, 2014
5aa5035
Remove API that exposes internal catalyst data structures.
marmbrus Mar 22, 2014
15e5265
Support for resolving mixed case fields from a reflected schema using…
marmbrus Mar 22, 2014
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Next Next commit
Attributes should be compared using exprId, not TreeNode.id.
  • Loading branch information
@marmbrus
marmbrus committed Mar 24, 2014
commit 9d99cb621ef0e16d680bccb328acf6e5af2b63c9
4 changes: 2 additions & 2 deletions sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveStrategies.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -73,11 +73,11 @@ trait HiveStrategies {
case p @ FilteredOperation(predicates, relation: MetastoreRelation)
if relation.isPartitioned =>

val partitionKeyIds = relation.partitionKeys.map(_.id).toSet
val partitionKeyIds = relation.partitionKeys.map(_.exprId).toSet

// Filter out all predicates that only deal with partition keys
val (pruningPredicates, otherPredicates) = predicates.partition {
_.references.map(_.id).subsetOf(partitionKeyIds)
_.references.map(_.exprId).subsetOf(partitionKeyIds)
}

val scan = HiveTableScan(
Expand Down