Skip to content

[SPARK-4923][REPL] Add Developer API to REPL to allow re-publishing the REPL jar #4034

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
chipsenkbeil wants to merge 5 commits into from
Closed

[SPARK-4923][REPL] Add Developer API to REPL to allow re-publishing the REPL jar #4034

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
925c112
Added DeveloperApi and Scaladocs to SparkIMain for Scala 2.10
Jan 14, 2015
26fd286
Refactored other Scala 2.10 classes and methods to be private/package…
Jan 14, 2015
6dc1ee2
Added missing method to expose error reporting flag
Jan 14, 2015
c1b88aa
Added DeveloperApi to public classes in repl
chipsenkbeil Jan 16, 2015
053ca75
Fixed failed build by adding missing DeveloperApi import
chipsenkbeil Jan 16, 2015
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 8 additions & 1 deletion repl/scala-2.10/src/main/scala/org/apache/spark/repl/SparkCommandLine.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,14 +19,21 @@ package org.apache.spark.repl

import scala.tools.nsc.{Settings, CompilerCommand}
import scala.Predef._
import org.apache.spark.annotation.DeveloperApi

/**
* Command class enabling Spark-specific command line options (provided by
* <i>org.apache.spark.repl.SparkRunnerSettings</i>).
*
* @example new SparkCommandLine(Nil).settings
*
* @param args The list of command line arguments
* @param settings The underlying settings to associate with this set of
* command-line options
*/
@DeveloperApi
class SparkCommandLine(args: List[String], override val settings: Settings)
Copy link
Contributor

@pwendell pwendell Jan 15, 2015

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should this be marked as a devloper API also, if it's exposed?

Copy link
Author

@chipsenkbeil chipsenkbeil Jan 15, 2015

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, do you use the DeveloperApi for entire classes? Or are you saying that since the purpose I described for SparkCommandLine was to retrieve settings?

I didn't mark SparkILoop, SparkIMain, SparkHelper (forced to be public due to packaging), SparkJLineCompletion, or SparkCommandLine as DeveloperApi on the class level. I was assuming that internal markings of DeveloperApi conveyed that. I can go back and do that if that's the way things are normally done.

Or, I can just add it to SparkCommandLine since it is the only one without any internal DeveloperApi marks.

Copy link
Contributor

@pwendell pwendell Jan 16, 2015

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Usually out of abundance of caution we mark it on the class level as well, even if everything bytecode-exposed inside of the class is also marked. We tend to err on the side of over communication with this.

extends CompilerCommand(args, settings) {

def this(args: List[String], error: String => Unit) {
this(args, new SparkRunnerSettings(error))
}
Expand Down
2 changes: 1 addition & 1 deletion repl/scala-2.10/src/main/scala/org/apache/spark/repl/SparkExprTyper.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ import scala.tools.nsc.ast.parser.Tokens.EOF

import org.apache.spark.Logging

trait SparkExprTyper extends Logging {
private[repl] trait SparkExprTyper extends Logging {
val repl: SparkIMain

import repl._
Expand Down
17 changes: 17 additions & 0 deletions repl/scala-2.10/src/main/scala/org/apache/spark/repl/SparkHelper.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,23 @@

package scala.tools.nsc

import org.apache.spark.annotation.DeveloperApi

// NOTE: Forced to be public (and in scala.tools.nsc package) to access the
// settings "explicitParentLoader" method

/**
* Provides exposure for the explicitParentLoader method on settings instances.
*/
@DeveloperApi
object SparkHelper {
/**
* Retrieves the explicit parent loader for the provided settings.
*
* @param settings The settings whose explicit parent loader to retrieve
*
* @return The Optional classloader representing the explicit parent loader
*/
@DeveloperApi
def explicitParentLoader(settings: Settings) = settings.explicitParentLoader
}
Loading