Skip to content

[SPARK-29837][SQL] PostgreSQL dialect: cast to boolean #26463

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
Ngone51 wants to merge 17 commits into from
Closed

[SPARK-29837][SQL] PostgreSQL dialect: cast to boolean #26463

Changes from 1 commit
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
75d51ba
postgresql dialect: cast to boolean
Ngone51 Nov 11, 2019
53c9a9b
fix PostgreSQLDialectQuerySuite
Ngone51 Nov 11, 2019
2ab1847
fix postgreSQL/boolean.sql
Ngone51 Nov 11, 2019
6c2c0c7
nullable should be false now
Ngone51 Nov 12, 2019
6ad100c
check unsupported types in checkInputDataTypes()
Ngone51 Nov 12, 2019
101a026
private[catalyst] -> protected[this]
Ngone51 Nov 12, 2019
be08daf
remove unused import
Ngone51 Nov 12, 2019
95caf40
return TypeCheckSuccess
Ngone51 Nov 12, 2019
8ebd5aa
use collect()
Ngone51 Nov 12, 2019
0b9a78d
use child.nullable
Ngone51 Nov 12, 2019
fda48c0
excludes unsupported types explicitly
Ngone51 Nov 12, 2019
36f60ee
protected[this] -> protected
Ngone51 Nov 12, 2019
b7c6baf
throw exception for PostgreCastToBoolean.ansiEnabled
Ngone51 Nov 12, 2019
b9711ae
include supportted types explicitly
Ngone51 Nov 13, 2019
68c7a13
adjust tests
Ngone51 Nov 13, 2019
999ec61
missing BooleanType
Ngone51 Nov 13, 2019
a178b79
leave cast( bool as bool) to optimizer
Ngone51 Nov 13, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
check unsupported types in checkInputDataTypes()
  • Loading branch information
@Ngone51
Ngone51 committed Nov 12, 2019
commit 6ad100c921ed6f1e52c2e191cced400b3a335206
15 changes: 6 additions & 9 deletions ...ain/scala/org/apache/spark/sql/catalyst/expressions/postgreSQL/PostgreCastToBoolean.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@
*/
package org.apache.spark.sql.catalyst.expressions.postgreSQL

import org.apache.spark.sql.catalyst.analysis.TypeCheckResult
import org.apache.spark.sql.catalyst.expressions.{CastBase, Expression, TimeZoneAwareExpression}
import org.apache.spark.sql.catalyst.expressions.codegen.Block._
import org.apache.spark.sql.catalyst.expressions.codegen.JavaCode
Copy link
Member Author

@Ngone51 Ngone51 Nov 12, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TODO: remove this unused import.

Expand All @@ -32,6 +33,11 @@ case class PostgreCastToBoolean(child: Expression, timeZoneId: Option[String])
override def withTimeZone(timeZoneId: String): TimeZoneAwareExpression =
copy(timeZoneId = Option(timeZoneId))

override def checkInputDataTypes(): TypeCheckResult = child.dataType match {
case StringType | IntegerType => super.checkInputDataTypes()
Copy link
Contributor

@cloud-fan cloud-fan Nov 12, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why call super.checkInputDataTypes() here? shall we just return success?

Copy link
Member Author

@Ngone51 Ngone51 Nov 12, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just want the StringType and IntegerType to do Cast.canCast() check in super.checkInputDataTypes(), though I know it's not necessary.

I'll return success directly.

case dt => throw new UnsupportedOperationException(s"cannot cast type $dt to boolean")
Copy link
Contributor

@cloud-fan cloud-fan Nov 12, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should follow other expressions and return TypeCheckResult.TypeCheckFailure here.

}

override def castToBoolean(from: DataType): Any => Any = from match {
case StringType =>
buildCast[UTF8String](_, str => {
Expand All @@ -44,10 +50,6 @@ case class PostgreCastToBoolean(child: Expression, timeZoneId: Option[String])
throw new IllegalArgumentException(s"invalid input syntax for type boolean: $s")
}
})
case TimestampType | DateType | LongType | ShortType |
ByteType | DecimalType() | DoubleType | FloatType =>
_ => throw new UnsupportedOperationException(s"cannot cast type $from to boolean")

case IntegerType =>
super.castToBoolean(from)
}
Expand All @@ -65,11 +67,6 @@ case class PostgreCastToBoolean(child: Expression, timeZoneId: Option[String])
throw new IllegalArgumentException("invalid input syntax for type boolean: $c");
}
"""
case TimestampType | DateType | LongType | ShortType |
ByteType | DecimalType() | DoubleType | FloatType =>
(c, evPrim, evNull) =>
val fromType = JavaCode.javaType(from)
code"""throw new UnsupportedOperationException("cannot cast type $fromType to boolean");"""

case IntegerType =>
super.castToBooleanCode(from)
Expand Down