Skip to content

[SPARK-8777] [SQL] Add random data generator test utilities to Spark SQL #7176

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
JoshRosen wants to merge 8 commits into from
Closed

[SPARK-8777] [SQL] Add random data generator test utilities to Spark SQL #7176

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
d2b4a4a
Add random data generator test utilities to Spark SQL.
JoshRosen Jul 2, 2015
ab76cbd
Move code to Catalyst package.
JoshRosen Jul 2, 2015
5acdd5c
Infinity and NaN are interesting.
JoshRosen Jul 2, 2015
b55875a
Generate doubles and floats over entire possible range.
JoshRosen Jul 2, 2015
0c20905
Initial attempt at using ScalaCheck.
JoshRosen Jul 2, 2015
89d86b1
Bump ScalaCheck version.
JoshRosen Jul 3, 2015
e0d7d49
Bump ScalaCheck version in LICENSE
JoshRosen Jul 3, 2015
f71634d
Roll back ScalaCheck usage
JoshRosen Jul 3, 2015
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Bump ScalaCheck version.
  • Loading branch information
@JoshRosen
JoshRosen committed Jul 3, 2015
commit 89d86b1f65f5db7044280d916b07d1622b4662a0
2 changes: 1 addition & 1 deletion pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -689,7 +689,7 @@
<dependency>
<groupId>org.scalacheck</groupId>
<artifactId>scalacheck_${scala.binary.version}</artifactId>
<version>1.11.3</version>
<version>1.12.4</version>
<scope>test</scope>
</dependency>
<dependency>
Expand Down
7 changes: 3 additions & 4 deletions sql/catalyst/src/test/scala/org/apache/spark/sql/RandomDataGenerator.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,13 +19,12 @@ package org.apache.spark.sql

import java.sql.Timestamp

import org.apache.spark.sql.types._
import org.scalacheck.{Arbitrary, Gen}

import org.apache.spark.sql.types._

/**
* Random data generators for Spark SQL DataTypes. These generators do not generate uniformly random
* values; instead, they're biased to return "interesting" values (such as maximum / minimum values)
* with higher probability.
* ScalaCheck random data generators for Spark SQL DataTypes.
*/
object RandomDataGenerator {

Expand Down
8 changes: 7 additions & 1 deletion sql/catalyst/src/test/scala/org/apache/spark/sql/types/DataTypeTestUtils.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,13 @@ object DataTypeTestUtils {
/**
* Instances of all [[AtomicType]]s.
*/
val atomicTypes: Set[DataType] = Set(BinaryType, StringType, TimestampType) ++ numericTypes
val atomicTypes: Set[DataType] = numericTypes ++ Set(
BinaryType,
BooleanType,
DateType,
StringType,
TimestampType
)

/**
* Instances of [[ArrayType]] for all [[AtomicType]]s. Arrays of these types may contain null.
Expand Down