Skip to content

Commit e6e4214

Browse files
committed
Upgrade python to Spark 2.3
1 parent 7162bb8 commit e6e4214

File tree

2 files changed

+5
-2
lines changed

2 files changed

+5
-2
lines changed

requirements.txt

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
coverage
2-
pyspark>=2.2.0
2+
pyspark>=2.3.0
33
pypandoc
44
scipy
55
numpy
@@ -8,3 +8,5 @@ unittest2>=1.0.0
88
pandas>=0.13
99
spacy
1010
future
11+
pyarrow==0.8.0
12+
pep8

setup.py

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -15,11 +15,12 @@
1515
description='Add additional ML algorithms to Spark',
1616
long_description=open('README.md').read(),
1717
install_requires=[
18-
'pyspark>=2.2.0',
18+
'pyspark>=2.3.0',
1919
'numpy', # Requires for PySpark ML
2020
'pandas',
2121
'spacy',
2222
'future',
23+
'pyarrow',
2324
],
2425
test_requires=[
2526
'nose==1.3.7',

0 commit comments

Comments
 (0)