Skip to content

[SPARK-23933][SQL] Add map_from_arrays function #21258

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
kiszk wants to merge 13 commits into from
Closed

[SPARK-23933][SQL] Add map_from_arrays function #21258

Changes from 1 commit
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
e4171e1
initial commit
kiszk May 7, 2018
95d92d8
fix pyspark test failure
kiszk May 7, 2018
1df6bb5
address review comments
kiszk May 7, 2018
2075770
address review comments
kiszk May 15, 2018
d5ff7be
fix pyspark test failure
kiszk May 16, 2018
4eee89d
fix pyspark test failure
kiszk May 17, 2018
7b66ab4
fix pyspark test failure
kiszk May 17, 2018
2fcbb80
fix pyspark test faiure
kiszk May 18, 2018
228fcc6
address review comments
kiszk Jun 8, 2018
6d53a96
address review comments
kiszk Jun 8, 2018
a4b3ec2
use show() to check pyspark result
kiszk Jun 10, 2018
a0b4ac5
fix pyspark test error
kiszk Jun 10, 2018
38d0868
address review comments
kiszk Jun 11, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
fix pyspark test faiure
  • Loading branch information
@kiszk
kiszk committed Jun 8, 2018
commit 2fcbb805801ebe94616e5c3ae2e4f76cd73e5f02
2 changes: 1 addition & 1 deletion python/pyspark/sql/functions.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1829,7 +1829,7 @@ def create_map_from_arrays(col1, col2):

>>> df = spark.createDataFrame([([2, 5], ["Alice", "Bob"])], ['k', 'v'])
>>> df.select(create_map_from_arrays(df.k, df.v).alias("map")).collect()
[Row(map={5: u'Bob', 2: u'Alice'})]
[Row(map={2: u'Alice', 5: u'Bob'})]
"""
sc = SparkContext._active_spark_context
return Column(sc._jvm.functions.map_from_arrays(_to_java_column(col1), _to_java_column(col2)))
Expand Down