Skip to content

[SPARK-8218][SQL] Add binary log math function #6725

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
viirya wants to merge 19 commits into from
Closed

[SPARK-8218][SQL] Add binary log math function #6725

Changes from 1 commit
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
f373bac
Add binary log expression.
viirya Jun 9, 2015
c795342
Merge remote-tracking branch 'upstream/master' into expr_binary_log
viirya Jun 10, 2015
c6c187f
For comments.
viirya Jun 10, 2015
21c3bfd
Fix scala style.
viirya Jun 10, 2015
605574d
Merge remote-tracking branch 'upstream/master' into expr_binary_log
viirya Jun 11, 2015
ebc9929
Let Logarithm accept one parameter too.
viirya Jun 11, 2015
23c54a3
Fix scala style.
viirya Jun 11, 2015
5b39c02
Merge remote-tracking branch 'upstream/master' into expr_binary_log
viirya Jun 12, 2015
3d75bfc
Fix scala style.
viirya Jun 12, 2015
1750034
Merge remote-tracking branch 'upstream/master' into expr_binary_log
viirya Jun 13, 2015
0634ef7
Merge remote-tracking branch 'upstream/master' into expr_binary_log
viirya Jun 14, 2015
db7dc38
Use ctor instead of companion object.
viirya Jun 14, 2015
bc89597
For comments.
viirya Jun 14, 2015
8cf37b7
For comments.
viirya Jun 14, 2015
6089d11
Remove unnecessary override.
viirya Jun 14, 2015
beed631
Merge remote-tracking branch 'upstream/master' into expr_binary_log
viirya Jun 14, 2015
fd01863
For comments.
viirya Jun 14, 2015
102070d
Round log result to better comparing in python test.
viirya Jun 14, 2015
bf96bd9
Compare log result in string.
viirya Jun 14, 2015
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Compare log result in string.
  • Loading branch information
@viirya
viirya committed Jun 14, 2015
commit bf96bd9ece95a41fb56151d2167b957fa357136e
8 changes: 4 additions & 4 deletions python/pyspark/sql/functions.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -408,11 +408,11 @@ def when(condition, value):
def log(col, base=math.e):
"""Returns the first argument-based logarithm of the second argument.

>>> df.select(log(df.age, 10.0).alias('ten')).map(lambda l: round(l.ten, 5)).collect()
[0.30103, 0.69897]
>>> df.select(log(df.age, 10.0).alias('ten')).map(lambda l: str(l.ten)[:7]).collect()
['0.30102', '0.69897']

>>> df.select(log(df.age).alias('e')).map(lambda l: round(l.e, 5)).collect()
[0.69315, 1.60944]
>>> df.select(log(df.age).alias('e')).map(lambda l: str(l.e)[:7]).collect()
['0.69314', '1.60943']
"""
sc = SparkContext._active_spark_context
jc = sc._jvm.functions.log(base, _to_java_column(col))
Expand Down