Skip to content

[SPARK-8378][Streaming]Add the Python API for Flume #6830

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
zsxwing wants to merge 16 commits into from
Closed

[SPARK-8378][Streaming]Add the Python API for Flume #6830

Changes from 1 commit
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
9f33873
Add the Python API for Flume
zsxwing Jun 12, 2015
0336579
Refactor Flume unit tests and also add tests for Python API
zsxwing Jun 19, 2015
4762c34
Fix the doc
zsxwing Jun 19, 2015
b8d5551
Merge branch 'master' into flume-python
zsxwing Jun 19, 2015
14ba0ff
Add flume-assembly for sbt building
zsxwing Jun 19, 2015
152364c
Fix the issue that StringIO doesn't work in Python 3
zsxwing Jun 20, 2015
01cbb3d
Add import sys
zsxwing Jun 20, 2015
ce85e83
Fix incompatible issues for Python 3
zsxwing Jun 21, 2015
b96b0de
Merge branch 'master' into flume-python
zsxwing Jun 23, 2015
7a55837
Add streaming_flume_assembly to run-tests.py
zsxwing Jun 23, 2015
f9ee681
Merge branch 'master' into flume-python
zsxwing Jun 29, 2015
9d5821e
Fix pyspark_core dependencies
zsxwing Jun 29, 2015
e93736b
Fix the test case for determine_modules_to_test
zsxwing Jun 29, 2015
0449723
Add sbt goal streaming-flume-assembly/assembly
zsxwing Jun 29, 2015
f1bf3c0
Address TD's comments
zsxwing Jun 30, 2015
78dfdac
Fix the compile error in the test code
zsxwing Jun 30, 2015
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Fix the compile error in the test code
  • Loading branch information
@zsxwing
zsxwing committed Jun 30, 2015
commit 78dfdac7216c3e68a42c6534f497289212a31a8a
2 changes: 1 addition & 1 deletion external/flume/src/test/scala/org/apache/spark/streaming/flume/FlumePollingStreamSuite.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -101,7 +101,7 @@ class FlumePollingStreamSuite extends SparkFunSuite with BeforeAndAfter with Log
val addresses = sinkPorts.map(port => new InetSocketAddress("localhost", port))
val flumeStream: ReceiverInputDStream[SparkFlumeEvent] =
FlumeUtils.createPollingStream(ssc, addresses, StorageLevel.MEMORY_AND_DISK,
utils.getEventsPerBatch, 5)
utils.eventsPerBatch, 5)
val outputBuffer = new ArrayBuffer[Seq[SparkFlumeEvent]]
with SynchronizedBuffer[Seq[SparkFlumeEvent]]
val outputStream = new TestOutputStream(flumeStream, outputBuffer)
Expand Down