Skip to content

[SPARK-8564][Streaming]Add the Python API for Kinesis #6955

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
zsxwing wants to merge 16 commits into from
Closed

[SPARK-8564][Streaming]Add the Python API for Kinesis #6955

Changes from 1 commit
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
4957c0b
Add the Python API for Kinesis
zsxwing Jun 23, 2015
add2beb
Merge branch 'master' into kinesis-python
zsxwing Jul 22, 2015
687446b
Fix the error message and the maven output path
zsxwing Jul 23, 2015
3da2601
Fix the kinesis folder
zsxwing Jul 23, 2015
e33d505
Merge remote-tracking branch 'origin/master' into kinesis-python
zsxwing Jul 23, 2015
466b425
Add python tests for Kinesis
zsxwing Jul 23, 2015
cc9d071
Fix the python test errors
zsxwing Jul 23, 2015
7c5cfb0
RUN_KINESIS_TESTS -> ENABLE_KINESIS_TESTS
zsxwing Jul 23, 2015
6c37395
Print stack trace for debug
zsxwing Jul 24, 2015
756a128
Merge branch 'master' into kinesis-python
zsxwing Jul 24, 2015
96670ff
Fix the compilation error after merging master
zsxwing Jul 24, 2015
fca416b
Fix wrong comparison
zsxwing Jul 24, 2015
5082d28
Fix the syntax error for Python 2.6
zsxwing Jul 26, 2015
32e6451
Merge remote-tracking branch 'origin/master' into kinesis-python
zsxwing Jul 30, 2015
455f7ea
Remove streaming_kinesis_asl_assembly module and simply add the sourc…
zsxwing Jul 30, 2015
e42e471
Merge branch 'master' into kinesis-python
zsxwing Jul 31, 2015
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Fix the kinesis folder
  • Loading branch information
@zsxwing
zsxwing committed Jul 23, 2015
commit 3da2601cd85a849a6549a404343dd98f22062628
2 changes: 1 addition & 1 deletion python/pyspark/streaming/tests.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -944,7 +944,7 @@ def search_flume_assembly_jar():

def search_kinesis_asl_assembly_jar():
SPARK_HOME = os.environ["SPARK_HOME"]
flume_assembly_dir = os.path.join(SPARK_HOME, "external/kinesis-asl-assembly")
flume_assembly_dir = os.path.join(SPARK_HOME, "extras/kinesis-asl-assembly")
jars = glob.glob(
os.path.join(flume_assembly_dir, "target/scala-*/spark-streaming-kinesis-asl-assembly-*.jar"))
if not jars:
Expand Down