Skip to content
Merged
Show file tree
Hide file tree
Changes from 22 commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
8ddb9fc
WIP: Plugin implementation of array_intersect using setIntersect, sor…
NVnavkumar Jun 28, 2022
0367a78
WIP: array_intersect
NVnavkumar Jun 29, 2022
7743f45
array_intersect implementation
NVnavkumar Jun 30, 2022
64946b2
implementation of 3 other array set operations
NVnavkumar Jul 4, 2022
ef54c68
add integration tests
NVnavkumar Jul 5, 2022
3362c08
working versions of array_difference and arrays_overlap
NVnavkumar Jul 5, 2022
728f15a
array_union passing tests but for one strange bug in GPU partitioning
NVnavkumar Jul 5, 2022
18ad1c0
Merge branch 'branch-22.08' of github.com:NVIDIA/spark-rapids into ar…
NVnavkumar Jul 5, 2022
3bfe106
cleanup arrays_overlap integration test
NVnavkumar Jul 5, 2022
c8e21b6
Add reference to issue filed regarding partitioning and collect()
NVnavkumar Jul 6, 2022
e7f978c
Merge branch 'branch-22.08' of github.com:NVIDIA/spark-rapids into ar…
NVnavkumar Jul 6, 2022
fab15a1
Add clarifying comment for GpuArraysOverlap
NVnavkumar Jul 6, 2022
f1003c4
Update integration_tests/src/main/python/array_test.py
NVnavkumar Jul 6, 2022
eb5e6f4
make generated arrays nullable in integration tests
NVnavkumar Jul 7, 2022
06af5eb
remove the plugin code for null return value handling since this is n…
NVnavkumar Jul 8, 2022
62fcdba
Merge branch 'branch-22.08' into array_intersect_string_array
NVnavkumar Jul 20, 2022
cd6beb2
Refactor to using GpuComplexTypeMergingExpression
NVnavkumar Jul 22, 2022
4587238
Add decimal_gens to testing here
NVnavkumar Jul 22, 2022
d2c8911
ensure all other special cases for floats and doubles are here except…
NVnavkumar Jul 22, 2022
893f7d0
Updated docs and xfail Double and Float tests on Spark versions < 3.1.3
NVnavkumar Jul 25, 2022
494ad54
Merge branch 'branch-22.08' into array_intersect_string_array
NVnavkumar Jul 25, 2022
3a83257
Updated references to methods in cudf and added some scalar tests
NVnavkumar Jul 26, 2022
5736e1e
Add incompat documentation, and update tests to handle pre Spark 3.1.…
NVnavkumar Aug 1, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions docs/configs.md
Original file line number Diff line number Diff line change
Expand Up @@ -161,11 +161,15 @@ Name | SQL Function(s) | Description | Default Value | Notes
<a name="sql.expression.And"></a>spark.rapids.sql.expression.And|`and`|Logical AND|true|None|
<a name="sql.expression.AnsiCast"></a>spark.rapids.sql.expression.AnsiCast| |Convert a column of one type of data into another type|true|None|
<a name="sql.expression.ArrayContains"></a>spark.rapids.sql.expression.ArrayContains|`array_contains`|Returns a boolean if the array contains the passed in key|true|None|
<a name="sql.expression.ArrayExcept"></a>spark.rapids.sql.expression.ArrayExcept|`array_except`|Returns an array of the elements in array1 but not in array2, without duplicates|true|None|
<a name="sql.expression.ArrayExists"></a>spark.rapids.sql.expression.ArrayExists|`exists`|Return true if any element satisfies the predicate LambdaFunction|true|None|
<a name="sql.expression.ArrayIntersect"></a>spark.rapids.sql.expression.ArrayIntersect|`array_intersect`|Returns an array of the elements in the intersection of array1 and array2, without duplicates|true|None|
<a name="sql.expression.ArrayMax"></a>spark.rapids.sql.expression.ArrayMax|`array_max`|Returns the maximum value in the array|true|None|
<a name="sql.expression.ArrayMin"></a>spark.rapids.sql.expression.ArrayMin|`array_min`|Returns the minimum value in the array|true|None|
<a name="sql.expression.ArrayRepeat"></a>spark.rapids.sql.expression.ArrayRepeat|`array_repeat`|Returns the array containing the given input value (left) count (right) times|true|None|
<a name="sql.expression.ArrayTransform"></a>spark.rapids.sql.expression.ArrayTransform|`transform`|Transform elements in an array using the transform function. This is similar to a `map` in functional programming|true|None|
<a name="sql.expression.ArrayUnion"></a>spark.rapids.sql.expression.ArrayUnion|`array_union`|Returns an array of the elements in the union of array1 and array2, without duplicates.|true|None|
<a name="sql.expression.ArraysOverlap"></a>spark.rapids.sql.expression.ArraysOverlap|`arrays_overlap`|Returns true if a1 contains at least a non-null element present also in a2. If the arrays have no common element and they are both non-empty and either of them contains a null element null is returned, false otherwise.|true|None|
<a name="sql.expression.ArraysZip"></a>spark.rapids.sql.expression.ArraysZip|`arrays_zip`|Returns a merged array of structs in which the N-th struct contains all N-th values of input arrays.|true|None|
<a name="sql.expression.Asin"></a>spark.rapids.sql.expression.Asin|`asin`|Inverse sine|true|None|
<a name="sql.expression.Asinh"></a>spark.rapids.sql.expression.Asinh|`asinh`|Inverse hyperbolic sine|true|None|
Expand Down
Loading