Skip to content

[SPARK-11450][SQL] Add Unsafe Row processing to Expand #9414

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
hvanhovell wants to merge 5 commits into from
Closed

[SPARK-11450][SQL] Add Unsafe Row processing to Expand #9414

hvanhovell wants to merge 5 commits into from

Conversation

@hvanhovell
Copy link
Contributor

@hvanhovell hvanhovell commented Nov 2, 2015

This PR enables the Expand operator to process and produce Unsafe Rows.

@marmbrus
Copy link
Contributor

marmbrus commented Nov 5, 2015

ok to test

@SparkQA
Copy link

SparkQA commented Nov 6, 2015

Test build #45160 has finished for PR 9414 at commit 43aeb0c.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

marmbrus
marmbrus reviewed Nov 6, 2015
View reviewed changes
sql/core/src/main/scala/org/apache/spark/sql/execution/Expand.scala Outdated
Copy link
Contributor

@marmbrus marmbrus Nov 6, 2015

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we do this inside of UnsafeProjection.create? It seems weird to have to remember to do it in each place we use it.

Copy link
Contributor Author

@hvanhovell hvanhovell Nov 6, 2015

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That would be much better. I'll make the change.

@SparkQA
Copy link

SparkQA commented Nov 6, 2015

Test build #45213 has finished for PR 9414 at commit 26f6f7e.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds the following public classes (experimental):\n * final class ShuffleSortDataFormat extends SortDataFormat<PackedRecordPointer, LongArray>\n * final class UnsafeSortDataFormat extends SortDataFormat<RecordPointerAndKeyPrefix, LongArray>\n

@marmbrus
Copy link
Contributor

marmbrus commented Nov 6, 2015

Thanks! Merging to master and 1.6.

@asfgit asfgit closed this in f328fed Nov 6, 2015
asfgit pushed a commit that referenced this pull request Nov 6, 2015
@hvanhovell @marmbrus
[SPARK-11450] [SQL] Add Unsafe Row processing to Expand
efa1e4a 
This PR enables the Expand operator to process and produce Unsafe Rows.

Author: Herman van Hovell <[email protected]>

Closes #9414 from hvanhovell/SPARK-11450.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@hvanhovell @marmbrus @SparkQA