Skip to content

[SPARK-8378][Streaming]Add the Python API for Flume #6830

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
zsxwing wants to merge 16 commits into from
Closed

[SPARK-8378][Streaming]Add the Python API for Flume #6830

Changes from 1 commit
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
9f33873
Add the Python API for Flume
zsxwing Jun 12, 2015
0336579
Refactor Flume unit tests and also add tests for Python API
zsxwing Jun 19, 2015
4762c34
Fix the doc
zsxwing Jun 19, 2015
b8d5551
Merge branch 'master' into flume-python
zsxwing Jun 19, 2015
14ba0ff
Add flume-assembly for sbt building
zsxwing Jun 19, 2015
152364c
Fix the issue that StringIO doesn't work in Python 3
zsxwing Jun 20, 2015
01cbb3d
Add import sys
zsxwing Jun 20, 2015
ce85e83
Fix incompatible issues for Python 3
zsxwing Jun 21, 2015
b96b0de
Merge branch 'master' into flume-python
zsxwing Jun 23, 2015
7a55837
Add streaming_flume_assembly to run-tests.py
zsxwing Jun 23, 2015
f9ee681
Merge branch 'master' into flume-python
zsxwing Jun 29, 2015
9d5821e
Fix pyspark_core dependencies
zsxwing Jun 29, 2015
e93736b
Fix the test case for determine_modules_to_test
zsxwing Jun 29, 2015
0449723
Add sbt goal streaming-flume-assembly/assembly
zsxwing Jun 29, 2015
f1bf3c0
Address TD's comments
zsxwing Jun 30, 2015
78dfdac
Fix the compile error in the test code
zsxwing Jun 30, 2015
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Add flume-assembly for sbt building
  • Loading branch information
@zsxwing
zsxwing committed Jun 19, 2015
commit 14ba0ffa3abcf106cb23edeb1ad85801dc8e8d6a
3 changes: 2 additions & 1 deletion dev/run-tests.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -273,7 +273,8 @@ def build_spark_sbt(hadoop_version):
build_profiles = get_build_profiles(hadoop_version, enable_hive_profiles=True)
sbt_goals = ["package",
"assembly/assembly",
"streaming-kafka-assembly/assembly"]
"streaming-kafka-assembly/assembly",
"streaming-flume-assembly/assembly"]
profiles_and_goals = build_profiles + sbt_goals

print "[info] Building Spark (w/Hive 0.13.1) using SBT with these arguments:",
Expand Down