Skip to content

[SPARK-29434][Core] Improve the MapStatuses Serialization Performance #26085

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
dbtsai wants to merge 18 commits into from
Closed

[SPARK-29434][Core] Improve the MapStatuses Serialization Performance #26085

Changes from 1 commit
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
78bbcf2
temp
dbtsai Sep 25, 2019
9714f39
tempMerge branch 'master' into mapStatus
dbtsai Oct 8, 2019
18e5bda
finish
dbtsai Oct 10, 2019
39502ed
tune test
dbtsai Oct 10, 2019
3958c01
ready to submit PR
dbtsai Oct 10, 2019
a4a807e
removed unused import
dbtsai Oct 10, 2019
d0c3532
revert
dbtsai Oct 10, 2019
ed08f2e
fix
dbtsai Oct 10, 2019
a601356
address feedback
dbtsai Oct 11, 2019
8dc8fad
update
dbtsai Oct 16, 2019
0bf182a
Fix styling and typo
dbtsai Oct 17, 2019
bd88abd
temp
dbtsai Oct 18, 2019
f184c4c
remove MapStatusesSerDeserBenchmark.scala
dbtsai Oct 18, 2019
bc6a14c
Merge branch 'master' into mapStatus
dbtsai Oct 18, 2019
5aadd8f
revert unneeded change
dbtsai Oct 18, 2019
d7fce82
Add JDK11
dongjoon-hyun Oct 18, 2019
7095b60
Add JDK8 result
dongjoon-hyun Oct 18, 2019
08c8fb2
Merge pull request #7 from dongjoon-hyun/PR-26085
dbtsai Oct 18, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
removed unused import
  • Loading branch information
@dbtsai
dbtsai committed Oct 10, 2019
commit a4a807edaadde98818afedb2cff3192a045b532c
1 change: 0 additions & 1 deletion core/src/main/scala/org/apache/spark/MapOutputTracker.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,6 @@ package org.apache.spark
import java.io._
import java.util.concurrent.{ConcurrentHashMap, LinkedBlockingQueue, ThreadPoolExecutor, TimeUnit}
import java.util.concurrent.locks.ReentrantReadWriteLock
import java.util.zip.{GZIPInputStream, GZIPOutputStream}

import com.github.luben.zstd.ZstdInputStream
import com.github.luben.zstd.ZstdOutputStream
Copy link
Member

@viirya viirya Oct 10, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I recall we put third party imports under java and scala.

Expand Down