-
Notifications
You must be signed in to change notification settings - Fork 29k
[SPARK-5214][Core] Add EventLoop and change DAGScheduler to an EventLoop #4016
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 1 commit
3b2e59c
1f73eac
55fb6f6
37f79c6
227bf33
460f7b3
dba35b2
5cfac83
aefa1ce
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
- Loading branch information
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -26,6 +26,9 @@ import org.apache.spark.Logging | |
| /** | ||
| * An event loop to receive events from the caller and process all events in the event thread. It | ||
| * will start an exclusive event thread to process all events. | ||
| * | ||
| * Note: The event queue will grow indefinitely. So subclasses should make sure `onReceive` can | ||
| * handle events in time to avoid the potential OOM. | ||
| */ | ||
| private[spark] abstract class EventLoop[E](name: String) extends Logging { | ||
|
|
||
|
|
@@ -84,17 +87,17 @@ private[spark] abstract class EventLoop[E](name: String) extends Logging { | |
| def isActive: Boolean = eventThread.isAlive | ||
|
|
||
| /** | ||
| * Invoke when `start()` is called. It's also invoked before the event thread starts. | ||
| * Invoked when `start()` is called but before the event thread starts. | ||
| */ | ||
| def onStart(): Unit = {} | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Change "invoke" to "invoked" here and below; otherwise, it sounds like it is the subclass which should invoke this.
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. is it supposed to be called from outside of the class? I mean, shall we tighten the access permission to protected?
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Good point. |
||
|
|
||
| /** | ||
| * Invoke when `stop()` is called and the event thread exits. | ||
| * Invoked when `stop()` is called and the event thread exits. | ||
| */ | ||
| def onStop(): Unit = {} | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. same here |
||
|
|
||
| /** | ||
| * Invoke in the event thread when polling events from the event queue. | ||
| * Invoked in the event thread when polling events from the event queue. | ||
| * | ||
| * Note: Should avoid calling blocking actions in `onReceive`, or the event thread will be blocked | ||
| * and cannot process events in time. If you want to call some blocking actions, run them in | ||
|
|
@@ -103,7 +106,8 @@ private[spark] abstract class EventLoop[E](name: String) extends Logging { | |
| def onReceive(event: E): Unit | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. since onReceive is supposed to be called only by eventThread, we'd better not expose it to the outside of the class |
||
|
|
||
| /** | ||
| * Invoke if `onReceive` throws any non fatal error. `onError` must not throw any non fatal error. | ||
| * Invoked if `onReceive` throws any non fatal error. Any non fatal error thrown from `onError` | ||
| * will be ignored. | ||
| */ | ||
| def onError(e: Throwable): Unit | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. same as onReceive |
||
|
|
||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
mark as DeveloperAPI?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is a private class. We only need to mark developer API's for exposed classes.