Skip to content

Commit 8e60d6e

Browse files
daugraphsrowen
authored andcommitted
[SPARK-36717][CORE] Incorrect order of variable initialization may lead incorrect behavior
### What changes were proposed in this pull request? Incorrect order of variable initialization may lead to incorrect behavior, related code: TorrentBroadcast.scala , TorrentBroadCast will get wrong checksumEnabled value after initialization, this may not be what we need, we can move L94 front of setConf(SparkEnv.get.conf) to avoid this. Supplement: Snippet 1 ```scala class Broadcast { def setConf(): Unit = { checksumEnabled = true } setConf() var checksumEnabled = false } println(new Broadcast().checksumEnabled) ``` output: ```scala false ``` Snippet 2 ```scala class Broadcast { var checksumEnabled = false def setConf(): Unit = { checksumEnabled = true } setConf() } println(new Broadcast().checksumEnabled) ``` output: ```scala true ``` ### Why are the changes needed? we can move L94 front of setConf(SparkEnv.get.conf) to avoid this. ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? No Closes #33957 from daugraph/branch0. Authored-by: daugraph <[email protected]> Signed-off-by: Sean Owen <[email protected]> (cherry picked from commit 65f6a7c) Signed-off-by: Sean Owen <[email protected]>
1 parent 0565d95 commit 8e60d6e

File tree

1 file changed

+4
-2
lines changed

1 file changed

+4
-2
lines changed

core/src/main/scala/org/apache/spark/broadcast/TorrentBroadcast.scala

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -73,6 +73,10 @@ private[spark] class TorrentBroadcast[T: ClassTag](obj: T, id: Long)
7373
/** Size of each block. Default value is 4MB. This value is only read by the broadcaster. */
7474
@transient private var blockSize: Int = _
7575

76+
77+
/** Whether to generate checksum for blocks or not. */
78+
private var checksumEnabled: Boolean = false
79+
7680
private def setConf(conf: SparkConf): Unit = {
7781
compressionCodec = if (conf.get(config.BROADCAST_COMPRESS)) {
7882
Some(CompressionCodec.createCodec(conf))
@@ -90,8 +94,6 @@ private[spark] class TorrentBroadcast[T: ClassTag](obj: T, id: Long)
9094
/** Total number of blocks this broadcast variable contains. */
9195
private val numBlocks: Int = writeBlocks(obj)
9296

93-
/** Whether to generate checksum for blocks or not. */
94-
private var checksumEnabled: Boolean = false
9597
/** The checksum for all the blocks. */
9698
private var checksums: Array[Int] = _
9799

0 commit comments

Comments
 (0)