Skip to content

[SPARK-8641][SPARK-8712][SQL] Native Spark Window Functions [WIP] #7715

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
hvanhovell wants to merge 19 commits into from
Closed

[SPARK-8641][SPARK-8712][SQL] Native Spark Window Functions [WIP] #7715

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
118edcf
Added WindowFunctions.
hvanhovell Jul 22, 2015
84401e7
Move to Native Spark UDAFs for window processing.
hvanhovell Jul 26, 2015
d348482
Rebase & make it compile again.
hvanhovell Jul 26, 2015
62c456f
Bug fixes....
hvanhovell Jul 26, 2015
bcff01b
Add NoOp to interpreted projections.
hvanhovell Jul 27, 2015
98c11b6
Bugfixes... All tests work now.
hvanhovell Jul 28, 2015
07d5655
Update after rebase. Removal of redundant code. Tweaks...
hvanhovell Jul 29, 2015
be5199c
Tiny touchup. Make Aggregate Processor a bit more DRYly
hvanhovell Jul 30, 2015
a21d67d
Rebase.
hvanhovell Jul 31, 2015
d27257c
Rebase. Performance improvements...
hvanhovell Jul 31, 2015
25c6f42
Rebase. Performance improvements...
hvanhovell Jul 31, 2015
1352d26
More performance tweaks.
hvanhovell Jul 31, 2015
d91bcae
Further tweaks
hvanhovell Aug 3, 2015
e792325
Rebase...
hvanhovell Aug 3, 2015
47b0923
Revert performance improvements (moved them to SPARK-9357).
hvanhovell Aug 6, 2015
e3091c6
Rebase/Revert Performance Tweaks
hvanhovell Aug 6, 2015
6836e4b
Revert change to BoundAttribute.
hvanhovell Aug 6, 2015
7deee38
Rebase. Removed OffsetRow. Replaced CompactBuffer with ArrayBuffer.
hvanhovell Aug 10, 2015
c579ca6
Removed left over from performance tuning. Cleanup some comments.
hvanhovell Aug 10, 2015
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Rebase & make it compile again.
  • Loading branch information
@hvanhovell
hvanhovell committed Aug 10, 2015
commit d348482679f6a7cde3c1cb5575eb0313d708dfd4
2 changes: 1 addition & 1 deletion sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/sets.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@ case class ReduceSetAlgebraic(left: Expression, right: AlgebraicAggregate)
@transient private[this] lazy val evaluate =
BindReferences.bindReference(right.evaluateExpression, right.bufferSchema.toAttributes)

@transient private[this] lazy val joinRow = new JoinedRow4
@transient private[this] lazy val joinRow = new JoinedRow

override def eval(input: InternalRow): Any = {
val result = left.eval(input).asInstanceOf[OpenHashSet[Any]]
Expand Down
1 change: 0 additions & 1 deletion ...catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/windowExpressions.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -400,7 +400,6 @@ abstract class AggregateWindowFunction extends AlgebraicAggregate with WindowFun
self: Product =>
override val frame = SpecifiedWindowFrame(RowFrame, UnboundedPreceding, CurrentRow)
override def dataType: DataType = IntegerType
override def foldable: Boolean = false
override def nullable: Boolean = false
override val mergeExpressions = Nil // TODO how to deal with this?
}
Expand Down