Skip to content

[SPARK-26118][Web UI] Introducing spark.ui.requestHeaderSize for setting HTTP requestHeaderSize #23090

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
attilapiros wants to merge 3 commits into from
Closed

[SPARK-26118][Web UI] Introducing spark.ui.requestHeaderSize for setting HTTP requestHeaderSize #23090

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
c6278be
Introducing spark.ui.requestHeaderSize
attilapiros Nov 19, 2018
81334f4
Remove unnecesary enters.
attilapiros Nov 19, 2018
a343f93
fix doc
attilapiros Nov 19, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions core/src/main/scala/org/apache/spark/internal/config/package.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -570,6 +570,12 @@ package object config {
.stringConf
.createOptional

private[spark] val UI_REQUEST_HEADER_SIZE =
ConfigBuilder("spark.ui.requestHeaderSize")
.doc("Value for HTTP request header size in bytes.")
.bytesConf(ByteUnit.BYTE)
.createWithDefaultString("8k")

private[spark] val EXTRA_LISTENERS = ConfigBuilder("spark.extraListeners")
.doc("Class names of listeners to add to SparkContext during initialization.")
.stringConf
Expand Down
6 changes: 4 additions & 2 deletions core/src/main/scala/org/apache/spark/ui/JettyUtils.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -356,13 +356,15 @@ private[spark] object JettyUtils extends Logging {

(connector, connector.getLocalPort())
}
val httpConfig = new HttpConfiguration()
httpConfig.setRequestHeaderSize(conf.get(UI_REQUEST_HEADER_SIZE).toInt)

// If SSL is configured, create the secure connector first.
val securePort = sslOptions.createJettySslContextFactory().map { factory =>
val securePort = sslOptions.port.getOrElse(if (port > 0) Utils.userPort(port, 400) else 0)
val secureServerName = if (serverName.nonEmpty) s"$serverName (HTTPS)" else serverName
val connectionFactories = AbstractConnectionFactory.getFactories(factory,
new HttpConnectionFactory())
new HttpConnectionFactory(httpConfig))

def sslConnect(currentPort: Int): (ServerConnector, Int) = {
newConnector(connectionFactories, currentPort)
Expand All @@ -377,7 +379,7 @@ private[spark] object JettyUtils extends Logging {

// Bind the HTTP port.
def httpConnect(currentPort: Int): (ServerConnector, Int) = {
newConnector(Array(new HttpConnectionFactory()), currentPort)
newConnector(Array(new HttpConnectionFactory(httpConfig)), currentPort)
}

val (httpConnector, httpPort) = Utils.startServiceOnPort[ServerConnector](port, httpConnect,
Expand Down
8 changes: 8 additions & 0 deletions docs/configuration.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -973,6 +973,14 @@ Apart from these, the following properties are also available, and may be useful
<br /><code>spark.com.test.filter1.param.name2=bar</code>
</td>
</tr>
<tr>
<td><code>spark.ui.requestHeaderSize</code></td>
<td>8k</td>
<td>
The HTTP request header size, in bytes unless otherwise specified.
This setting applied for the Spark History Server too.
<td>
</tr>
</table>

### Compression and Serialization
Expand Down