Skip to content

[SPARK-33504][CORE] The application log in the Spark history server contains sensitive attributes should be redacted #30446

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
echohlne wants to merge 21 commits into from
Closed

[SPARK-33504][CORE] The application log in the Spark history server contains sensitive attributes should be redacted #30446

Changes from 1 commit
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
b1eb0d8
Update EventLoggingListener.scala
echohlne Nov 20, 2020
9bd82c9
Update EventLoggingListenerSuite.scala
echohlne Nov 20, 2020
6554d68
Update EventLoggingListener.scala
echohlne Nov 20, 2020
cd81888
Update EventLoggingListenerSuite.scala
echohlne Nov 20, 2020
a60e22e
Update EventLoggingListener.scala
echohlne Nov 20, 2020
875a32d
Update EventLoggingListener.scala
echohlne Nov 20, 2020
0d3c6b3
Update EventLoggingListenerSuite.scala
echohlne Nov 20, 2020
f575453
Update EventLoggingListener.scala
echohlne Nov 20, 2020
f2e05fa
Update EventLoggingListener.scala
echohlne Nov 20, 2020
33b6e24
Update EventLoggingListenerSuite.scala
echohlne Nov 20, 2020
2e3f931
Update EventLoggingListener.scala
echohlne Nov 21, 2020
f03fdef
Update EventLoggingListenerSuite.scala
echohlne Nov 21, 2020
fdb305c
Update EventLoggingListenerSuite.scala
echohlne Nov 21, 2020
26440c3
Update EventLoggingListenerSuite.scala
echohlne Nov 21, 2020
84a1a28
Update EventLoggingListenerSuite.scala
echohlne Nov 21, 2020
a73f527
Update EventLoggingListenerSuite.scala
echohlne Nov 21, 2020
1a4e5c2
Update EventLoggingListenerSuite.scala
echohlne Nov 21, 2020
cd4a184
Update EventLoggingListenerSuite.scala
echohlne Nov 21, 2020
805d704
Update EventLoggingListenerSuite.scala
echohlne Nov 22, 2020
c231252
Update EventLoggingListener.scala
echohlne Nov 22, 2020
effbead
Update EventLoggingListenerSuite.scala
echohlne Nov 22, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Update EventLoggingListener.scala
  • Loading branch information
@echohlne
echohlne authored Nov 20, 2020
commit a60e22eb921db3f57eaa8d437c39701d6fed0356
3 changes: 2 additions & 1 deletion core/src/main/scala/org/apache/spark/scheduler/EventLoggingListener.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -287,7 +287,8 @@ private[spark] class EventLoggingListener(
}
val redactedProperties = new Properties

// only properties in sparkConf need to be redacted.
// properties may contain some custom local properties such as stage/job description
// only properties in sparkConf need to be redacted,
val (globalProperties, localProperties) = properties.asScala.toSeq.partition {
case (key, _) => sparkConf.contains(key)
}
Expand Down