Skip to content

[SPARK-4233] [SQL] UDAF Interface Refactoring #5542

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
chenghao-intel wants to merge 20 commits into from
Closed

[SPARK-4233] [SQL] UDAF Interface Refactoring #5542

Changes from 1 commit
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
440b689
migrate to support both version of UDAF
chenghao-intel Apr 11, 2015
7fb0662
Update the unit test to comment out the not support ones
chenghao-intel Apr 11, 2015
f118ffc
update the interface name
chenghao-intel Apr 13, 2015
f0b9ec0
change the update method from Any to Row
chenghao-intel Apr 15, 2015
bee0f95
move the distinct into the udaf
chenghao-intel Apr 15, 2015
760164e
simpify the aggregate expression by uing the Projection
chenghao-intel Apr 16, 2015
0849ca3
revert the uncessary changes
chenghao-intel Apr 16, 2015
472a440
Add Unit test
chenghao-intel Apr 16, 2015
241aee1
Add some doc
chenghao-intel Apr 16, 2015
de96a13
style issues
chenghao-intel Apr 16, 2015
483b381
more style issues
chenghao-intel Apr 16, 2015
58b1481
fix bug in the for unit test
chenghao-intel Apr 16, 2015
39a6243
use BufferAndKey class manully maitain the MutableRow
chenghao-intel Apr 17, 2015
5b01518
fix bug of with BufferAndKeys
chenghao-intel Apr 17, 2015
feac4d0
Add golden files
chenghao-intel Apr 17, 2015
ec7deaa
enable more unit test
chenghao-intel Apr 21, 2015
393b0d1
disable the codegen for aggregate2 in unit test
chenghao-intel Apr 21, 2015
021431f
Add more unit test
chenghao-intel Apr 24, 2015
8ad5fc5
rebase to the latest master
chenghao-intel May 18, 2015
68dd625
rebase again
chenghao-intel Jun 8, 2015
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
fix bug of with BufferAndKeys
  • Loading branch information
@chenghao-intel
chenghao-intel committed Jun 8, 2015
commit 5b015184fef3ef3a414f84238dfd04ccb1246ab2
6 changes: 3 additions & 3 deletions sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate2/Aggregate.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,7 @@ sealed class BufferAndKey(leftLen: Int, rightLen: Int)
// copy the keys to the last
var idx = leftLen
var idx2 = 0
while (idx < keys.length) {
while (idx2 < keys.length) {
this.values(idx) = keys(idx2)
idx2 += 1
idx += 1
Expand Down Expand Up @@ -210,7 +210,7 @@ case class AggregatePreShuffle(
val keys = groupByProjection(currentRow)
results(keys) match {
case null =>
val buffer = new BufferAndKey(output.length, keys)
val buffer = new BufferAndKey(buffersSchema.length, keys)
// update the aggregate buffers
var idx = 0
while (idx < aggregates.length) {
Expand Down Expand Up @@ -426,7 +426,7 @@ case class DistinctAggregate(
val keys = groupByProjection(currentRow)
results(keys) match {
case null =>
val buffer = new BufferAndKey(aggregates.length, keys)
val buffer = new BufferAndKey(buffersSchema.length, keys)
val seens = new Array[JSet[Any]](aggregates.length)

// update the aggregate buffers
Expand Down