Skip to content
Open
Changes from 1 commit
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
53f77ac
start thinking about cleanup timers
holdenk Sep 30, 2024
86a9f7f
Start a bit on putting in block tracking.
holdenk Oct 25, 2024
74c2608
Test compiles
holdenk Dec 2, 2024
94c290a
Add the Block TTL Integration suite
holdenk Dec 2, 2024
7bc2f7a
Increase logging a little bit.
holdenk Dec 3, 2024
cd30325
Make the TTL cleaners clean and add some simple tests
holdenk Dec 5, 2024
0f23de4
Switch to using MDC logging
holdenk Dec 5, 2024
ce72a3c
hmmm fails in CI maybe our TTL is too tight
holdenk Dec 14, 2024
0fd8744
Bump TTL a bit
holdenk Dec 15, 2024
5b6ae44
Use eventually instead of sleep
holdenk Dec 16, 2024
92ae9af
Back out un-needed change to ShuffledRDD.scala
sfc-gh-hkarau Nov 3, 2025
c7dbcdd
Add a note about why JHashMap
sfc-gh-hkarau Nov 3, 2025
feec34b
Add a version 4.1.0 to the new config options.
sfc-gh-hkarau Nov 3, 2025
eb4828c
CR feedback
sfc-gh-hkarau Nov 13, 2025
c5aa83c
Merge branch 'master' into SPARK-49788-Restore-Cleaner-TTL-Functionality
sfc-gh-hkarau Nov 13, 2025
0354749
Merge branch 'master' into SPARK-49788-Restore-Cleaner-TTL-Functionality
sfc-gh-hkarau Nov 14, 2025
f474dc0
Deal with the interruption excdeption from shutdownnow
sfc-gh-hkarau Nov 15, 2025
3a37726
Merge branch 'master' into SPARK-49788-Restore-Cleaner-TTL-Functionality
sfc-gh-hkarau Nov 27, 2025
53a65c6
Merge branch 'master' into SPARK-49788-Restore-Cleaner-TTL-Functionality
sfc-gh-hkarau Dec 18, 2025
4ac8a19
Merge branch 'master' into SPARK-49788-Restore-Cleaner-TTL-Functionality
sfc-gh-hkarau Dec 19, 2025
afc9a6a
Merge branch 'master' into SPARK-49788-Restore-Cleaner-TTL-Functionality
sfc-gh-hkarau Dec 22, 2025
f3a7531
Reduce the TTL in the tests so they don't take so long, move the time…
sfc-gh-hkarau Dec 22, 2025
34090b1
Increase TTL so the blocks don't get cleaned before the test has a ch…
sfc-gh-hkarau Dec 23, 2025
1ca2378
Merge branch 'master' into SPARK-49788-Restore-Cleaner-TTL-Functionality
sfc-gh-hkarau Dec 29, 2025
e04d2e1
Bump up the blockTTL for testing since CI seems to cleanup the blocks…
sfc-gh-hkarau Dec 29, 2025
f9f66e1
hmmm
sfc-gh-hkarau Dec 29, 2025
c7fc32d
Merge branch 'master' into SPARK-49788-Restore-Cleaner-TTL-Functionality
sfc-gh-hkarau Jan 12, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Add a version 4.1.0 to the new config options.
  • Loading branch information
sfc-gh-hkarau committed Nov 3, 2025
commit feec34b21f90854df7374de9d08706755d19a718
Original file line number Diff line number Diff line change
Expand Up @@ -2911,6 +2911,7 @@ package object config {
" GC on driver which with DataFrames or RDDs at the global scope will not occur. " +
"This must be configured before starting the SparkContext (e.g. can not be added to a" +
"a running Spark instance.)")
.version("4.1.0")
.timeConf(TimeUnit.MILLISECONDS)
.createOptional

Expand All @@ -2920,6 +2921,7 @@ package object config {
"after GC on driver which with DataFrames or RDDs at the global scope will not occur." +
"This must be configured before starting the SparkContext (e.g. can not be added to a" +
"a running Spark instance.)")
.version("4.1.0")
.timeConf(TimeUnit.MILLISECONDS)
.createOptional
}