Skip to content

[SPARK-17297] [DOCS] Clarify window/slide duration as absolute time, not relative to a calendar #15142

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
srowen wants to merge 1 commit into from
Closed

[SPARK-17297] [DOCS] Clarify window/slide duration as absolute time, not relative to a calendar #15142

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 6 additions & 2 deletions R/pkg/R/functions.R
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2713,11 +2713,15 @@ setMethod("from_unixtime", signature(x = "Column"),
#' @param x a time Column. Must be of TimestampType.
#' @param windowDuration a string specifying the width of the window, e.g. '1 second',
#' '1 day 12 hours', '2 minutes'. Valid interval strings are 'week',
#' 'day', 'hour', 'minute', 'second', 'millisecond', 'microsecond'.
#' 'day', 'hour', 'minute', 'second', 'millisecond', 'microsecond'. Note that
#' the duration is a fixed length of time, and does not vary over time
#' according to a calendar. For example, '1 day' always means 86,400,000
#' milliseconds, not a calendar day.
#' @param slideDuration a string specifying the sliding interval of the window. Same format as
#' \code{windowDuration}. A new window will be generated every
#' \code{slideDuration}. Must be less than or equal to
#' the \code{windowDuration}.
#' the \code{windowDuration}. This duration is likewise absolute, and does not
#' vary according to a calendar.
#' @param startTime the offset with respect to 1970-01-01 00:00:00 UTC with which to start
#' window intervals. For example, in order to have hourly tumbling windows
#' that start 15 minutes past the hour, e.g. 12:15-13:15, 13:15-14:15... provide
Expand Down
15 changes: 11 additions & 4 deletions sql/core/src/main/scala/org/apache/spark/sql/functions.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2606,12 +2606,15 @@ object functions {
* The time column must be of TimestampType.
* @param windowDuration A string specifying the width of the window, e.g. `10 minutes`,
* `1 second`. Check [[org.apache.spark.unsafe.types.CalendarInterval]] for
* valid duration identifiers.
* valid duration identifiers. Note that the duration is a fixed length of
* time, and does not vary over time according to a calendar. For example,
* `1 day` always means 86,400,000 milliseconds, not a calendar day.
* @param slideDuration A string specifying the sliding interval of the window, e.g. `1 minute`.
* A new window will be generated every `slideDuration`. Must be less than
* or equal to the `windowDuration`. Check
* [[org.apache.spark.unsafe.types.CalendarInterval]] for valid duration
* identifiers.
* identifiers. This duration is likewise absolute, and does not vary
* according to a calendar.
* @param startTime The offset with respect to 1970-01-01 00:00:00 UTC with which to start
* window intervals. For example, in order to have hourly tumbling windows that
* start 15 minutes past the hour, e.g. 12:15-13:15, 13:15-14:15... provide
Expand Down Expand Up @@ -2660,11 +2663,15 @@ object functions {
* The time column must be of TimestampType.
* @param windowDuration A string specifying the width of the window, e.g. `10 minutes`,
* `1 second`. Check [[org.apache.spark.unsafe.types.CalendarInterval]] for
* valid duration identifiers.
* valid duration identifiers. Note that the duration is a fixed length of
* time, and does not vary over time according to a calendar. For example,
* `1 day` always means 86,400,000 milliseconds, not a calendar day.
* @param slideDuration A string specifying the sliding interval of the window, e.g. `1 minute`.
* A new window will be generated every `slideDuration`. Must be less than
* or equal to the `windowDuration`. Check
* [[org.apache.spark.unsafe.types.CalendarInterval]] for valid duration.
* [[org.apache.spark.unsafe.types.CalendarInterval]] for valid duration
* identifiers. This duration is likewise absolute, and does not vary
* according to a calendar.
*
* @group datetime_funcs
* @since 2.0.0
Expand Down