[SPARK-5932][CORE] Use consistent naming for size properties#5574

Closed

ilganeli wants to merge 49 commits intoapache:masterfrom

ilganeli:SPARK-5932

Commits on Apr 13, 2015

Updated SparkConf class to add getOrCreate method. Started test suite implementation
Ilya Ganelin
committed

Commits on Apr 14, 2015

Spacing fix
Ilya Ganelin
committed
[SPARK-6703] Added test to ensure that getOrCreate both allows creation, retrieval, and a second context if desired
Ilya Ganelin
committed
Replaced match with if
Ilya Ganelin
committed
Fixed some bugs in test code
Ilya Ganelin
committed
Changed activeContext to AtomicReference
Ilya Ganelin
committed
Got rid of unecessary option for AtomicReference
Ilya Ganelin
committed
Updated comments and added MiMa Exclude
Ilya Ganelin
committed
[SPARK-6703] Documentation fixes
Ilya Ganelin
committed
Doc updates and code cleanup
Ilya Ganelin
committed
Made getOrCreate synchronized
Ilya Ganelin
committed
Added ref equality check
Ilya Ganelin
committed

Commits on Apr 17, 2015

Closing second spark context
Ilya Ganelin
committed

Commits on Apr 18, 2015

Merge remote-tracking branch 'upstream/master' into SPARK-5932
Ilya Ganelin
committed
[SPARK-5932] Added byte string conversion to Jav utils
Ilya Ganelin
committed
[SPARK-5932] Added unit tests for ByteString conversion
Ilya Ganelin
committed
[SPARK-5932] Added unit tests for unit conversion
Ilya Ganelin
committed
[SPARK-5932] Updated memoryStringToMb to use new interfaces
Ilya Ganelin
committed
[SPARK-5932] Simplified ByteUnit code, switched to using longs. Updated docs to clarify that we use kibi, mebi etc instead of kilo, mega
Ilya Ganelin
committed
[SPARK-5932] Updated to use bibibytes in method names. Updated spark.kryoserializer.buffer.mb and spark.reducer.maxMbInFlight
Ilya Ganelin
committed
[SPARK-5932] Updated spark.kryoserializer.buffer.max
Ilya Ganelin
committed
[SPARK-5932] Updated spark.shuffle.file.buffer.kb
Ilya Ganelin
committed
[SPARK-5932] Updated spark.broadcast.blockSize
Ilya Ganelin
committed
[SPARK-5932] Updated spark.executor.logs.rolling.size.maxBytes
Ilya Ganelin
committed
[SPARK-5932] Updated spark.io.compression.snappy.block.size
Ilya Ganelin
committed
[SPARK-5932] Updated spark.broadcast.blockSize and spark.storage.memoryMapThreshold
Ilya Ganelin
committed
[SPARK-5932] Updated spark.shuffle.file.buffer
Ilya Ganelin
committed
[SPARK-5932] Finished documentation updates
Ilya Ganelin
committed
[SPARK-5932] Converted some longs to ints
Ilya Ganelin
committed
Minor formatting
Ilya Ganelin
committed
Merge remote-tracking branch 'upstream/master' into SPARK-5932
Ilya Ganelin
committed

Commits on Apr 19, 2015

Commits on Apr 21, 2015

Updates to code
Ilya Ganelin
committed
Simplified byteUnit class
Ilya Ganelin
committed
Empty lines
Ilya Ganelin
committed
Resolved merge conflict
Ilya Ganelin
committed
[SPARK-5932] Fixing error in KryoSerializer
Ilya Ganelin
committed
Added smoother handling of fractional values for size parameters. This now throws an exception and added a warning for old spark.kryoserializer.buffer
Ilya Ganelin
committed
Fixed error in pattern matching for doubles
Ilya Ganelin
committed

Commits on Apr 22, 2015

resolving merge conflict
Ilya Ganelin
committed
Fixed some nits. Added automatic conversion of old paramter for kryoserializer.mb to new values.
Ilya Ganelin
committed
Made MAX private
Ilya Ganelin
committed
Simplified fraction matches
Ilya Ganelin
committed

Commits on Apr 23, 2015

[SPARK-5932] Added much improved overflow handling. Can now handle sizes up to Long.MAX_VALUE Petabytes instead of being capped at Long.MAX_VALUE Bytes
Ilya Ganelin
committed

Commits on Apr 24, 2015

Got rid of floating point math
Ilya Ganelin
committed
Scala style
Ilya Ganelin
committed
Whitespace fix
Ilya Ganelin
committed
Nit fixes
Ilya Ganelin
committed