Skip to content

[SPARK-22614] Dataset API: repartitionByRange(...) #19828

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
adrian-ionescu wants to merge 12 commits into from
Closed

[SPARK-22614] Dataset API: repartitionByRange(...) #19828

Changes from 1 commit
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
08527b0
repartitionByRange() + repartition() with SortOrders
adrian-ionescu Nov 27, 2017
950b3dc
avoid changing semantics for .repartition()
adrian-ionescu Nov 27, 2017
671e9e4
review feedback: val partition = ... + unit test
adrian-ionescu Nov 27, 2017
fe690fc
move test to DataFrameSuite + minor ws
adrian-ionescu Nov 28, 2017
66b192d
handle empty partition-by expr list as RoundRobin
adrian-ionescu Nov 28, 2017
012d617
update docs
adrian-ionescu Nov 29, 2017
60ec0e3
fix typo: hash -> range
adrian-ionescu Nov 29, 2017
e0bda1d
Revert "fix typo: hash -> range"
adrian-ionescu Nov 30, 2017
2f0f8f4
Revert "update docs"
adrian-ionescu Nov 30, 2017
f6cd388
Revert "handle empty partition-by expr list as RoundRobin"
adrian-ionescu Nov 30, 2017
67ca139
avoid changing repartition() and make repartitionByRange() throw on e…
adrian-ionescu Nov 30, 2017
012baa0
fix triple quotes
adrian-ionescu Nov 30, 2017
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
fix typo: hash -> range
  • Loading branch information
@adrian-ionescu
adrian-ionescu committed Nov 29, 2017
commit 60ec0e39f04ce4f3a1d5e539ead8ed408f443430
2 changes: 1 addition & 1 deletion sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2760,7 +2760,7 @@ class Dataset[T] private[sql](
}

/**
* Returns a new Dataset that is hash partitioned by the given expressions into `numPartitions`.
* Returns a new Dataset that is range partitioned by the given expressions into `numPartitions`.
* If no expressions are specified, round robin partitioning is used.
*
* @group typedrel
Expand Down