Skip to content

[SPARK-16283][SQL] Implements percentile_approx aggregation function which supports partial aggregation. #14868

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
clockfly wants to merge 6 commits into from
Closed

[SPARK-16283][SQL] Implements percentile_approx aggregation function which supports partial aggregation. #14868

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
6be2ebe
percentile approximate
clockfly Aug 19, 2016
3dda1ae
fix UT
clockfly Aug 30, 2016
96bd5a4
address comments
clockfly Aug 30, 2016
539d693
address comments
clockfly Aug 30, 2016
1965b1a
address comments
clockfly Aug 30, 2016
3f08c02
address wenchen's comment
clockfly Aug 31, 2016
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
address comments
  • Loading branch information
@clockfly
clockfly committed Aug 30, 2016
commit 1965b1a918633f0d9b94b0a245605592cddbeb9b
2 changes: 1 addition & 1 deletion ...ain/scala/org/apache/spark/sql/catalyst/expressions/aggregate/ApproximatePercentile.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ import org.apache.spark.sql.types._
*
* @param child child expression that can produce column value with `child.eval(inputRow)`
* @param percentageExpression Expression that represents a single percentage value or
* a array of percentage values. Each percentage value must be between
* an array of percentage values. Each percentage value must be between
* 0.0 and 1.0.
* @param accuracyExpression Integer literal expression of approximation accuracy. Higher value
* yields better accuracy, the default value is
Expand Down
6 changes: 3 additions & 3 deletions ...cala/org/apache/spark/sql/catalyst/expressions/aggregate/ApproximatePercentileSuite.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -170,7 +170,7 @@ class ApproximatePercentileSuite extends SparkFunSuite {
val defaultAccuracy = ApproximatePercentile.DEFAULT_PERCENTILE_ACCURACY
// sql, single percentile
assertEqual(
"percentile_approx(`a`, 0.5D, 1000)",
s"percentile_approx(`a`, 0.5D, $defaultAccuracy)",
new ApproximatePercentile("a".attr, percentageExpression = Literal(0.5D)).sql: String)

// sql, array of percentile
Expand All @@ -183,7 +183,7 @@ class ApproximatePercentileSuite extends SparkFunSuite {

// sql(isDistinct = false), single percentile
assertEqual(
"percentile_approx(`a`, 0.5D, 1000)",
s"percentile_approx(`a`, 0.5D, $defaultAccuracy)",
new ApproximatePercentile("a".attr, percentageExpression = Literal(0.5D))
.sql(isDistinct = false))

Expand All @@ -197,7 +197,7 @@ class ApproximatePercentileSuite extends SparkFunSuite {

// sql(isDistinct = true), single percentile
assertEqual(
"percentile_approx(DISTINCT `a`, 0.5D, 1000)",
s"percentile_approx(DISTINCT `a`, 0.5D, $defaultAccuracy)",
new ApproximatePercentile("a".attr, percentageExpression = Literal(0.5D))
.sql(isDistinct = true))

Expand Down