Skip to content

Commit 4257a94

Browse files
ajithmesrowen
authored andcommitted
[SPARK-30360][UI] Avoid Redact classpath entries in History Server UI
Currently SPARK history server display the classpath entries in the Environment tab with classpaths redacted, this is because EventLog file has the entry values redacted while writing. But when same is seen from a running application UI, its seen that it is not redacted. Classpath entries redact is not needed and can be avoided ### What changes were proposed in this pull request? Event logs will not redect the classpath entries ### Why are the changes needed? Redact of classpath entries is not needed ### Does this PR introduce any user-facing change? NO ### How was this patch tested? Tested manually to verify on UI Closes apache#27016 from ajithme/redactui. Authored-by: Ajith <ajith2489@gmail.com> Signed-off-by: Sean Owen <srowen@gmail.com>
1 parent 0496413 commit 4257a94

File tree

1 file changed

+4
-2
lines changed

1 file changed

+4
-2
lines changed

core/src/main/scala/org/apache/spark/scheduler/EventLoggingListener.scala

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -255,8 +255,10 @@ private[spark] class EventLoggingListener(
255255
// ...
256256
// where jvmInformation, sparkProperties, etc. are sequence of tuples.
257257
// We go through the various of properties and redact sensitive information from them.
258-
val redactedProps = event.environmentDetails.map{ case (name, props) =>
259-
name -> Utils.redact(sparkConf, props)
258+
val noRedactProps = Seq("Classpath Entries")
259+
val redactedProps = event.environmentDetails.map {
260+
case (name, props) if noRedactProps.contains(name) => name -> props
261+
case (name, props) => name -> Utils.redact(sparkConf, props)
260262
}
261263
SparkListenerEnvironmentUpdate(redactedProps)
262264
}

0 commit comments

Comments
 (0)