[SPARK-10400] [SQL] Deprecates SQLConf.PARQUET_FOLLOW_PARQUET_FORMAT_SPEC#7

Closed

liancheng wants to merge 112 commits intomasterfrom

spark-10400/deprecate-follow-parquet-format-spec

Commits on Sep 1, 2015

Commits on Sep 2, 2015

[SPARK-7336] [HISTORYSERVER] Fix bug that applications status incorrect on JobHistory UI.

ArcherShao
authored and
Marcelo Vanzin
committed
[SPARK-10034] [SQL] add regression test for Sort on Aggregate

cloud-fan
authored and
marmbrus
committed
[SPARK-10389] [SQL] support order by non-attribute grouping expression on Aggregate

cloud-fan
authored and
marmbrus
committed
[SPARK-10004] [SHUFFLE] Perform auth checks when clients read shuffle data.
Marcelo Vanzin
committed
[SPARK-10417] [SQL] Iterating through Column results in infinite loop

0x0FFF
authored and
davies
committed

Commits on Sep 3, 2015

[SPARK-10422] [SQL] String column in InMemoryColumnarCache needs to override clone method

yhuai
authored and
davies
committed
[SPARK-9723] [ML] params getordefault should throw more useful error

holdenk
authored and
mengxr
committed
[SPARK-5945] Spark should not retry a stage infinitely on a FetchFailedException

Ilya Ganelin
authored and
Andrew Or
committed
[SPARK-8707] RDD#toDebugString fails if any cached RDD has invalid partitions

navis
authored and
Andrew Or
committed
Removed code duplication in ShuffleBlockFetcherIterator

eracah
authored and
Andrew Or
committed
[SPARK-10247] [CORE] improve readability of a test case in DAGSchedulerSuite

squito
authored and
Andrew Or
committed
[SPARK-10379] preserve first page in UnsafeShuffleExternalSorter

Davies Liu
authored and
Andrew Or
committed
[SPARK-10411] [SQL] Move visualization above explain output and hide explain by default

zsxwing
authored and
Andrew Or
committed
[SPARK-10332] [CORE] Fix yarn spark executor validation

holdenk
authored and
srowen
committed
[SPARK-9596] [SQL] treat hadoop classes as shared one in IsolatedClientLoader

WangTaoTheTonic
authored and
marmbrus
committed
[SPARK-8951] [SPARKR] support Unicode characters in collect()

CHOIJAEHONG
authored and
shivaram
committed
[SPARK-10432] spark.port.maxRetries documentation is unclear

Tom Graves
authored and
Andrew Or
committed
[SPARK-10431] [CORE] Fix intermittent test failure. Wait for event queue to be clear

robbins
authored and
Andrew Or
committed
[SPARK-9869] [STREAMING] Wait for all event notifications before asserting results

robbins
authored and
Andrew Or
committed
[SPARK-9672] [MESOS] Don’t include SPARK_ENV_LOADED when passing env vars

pashields
authored and
Andrew Or
committed
[SPARK-10430] [CORE] Added hashCode methods in AccumulableInfo and RDDOperationScope

Vinod K C
authored and
Andrew Or
committed
[SPARK-9591] [CORE] Job may fail for exception during getting remote block

jeanlyn
authored and
Andrew Or
committed
[SPARK-10435] Spark submit should fail fast for Mesos cluster mode with R
Andrew Or
committed
[SPARK-10421] [BUILD] Exclude curator artifacts from tachyon dependencies.
Marcelo Vanzin
committed

Commits on Sep 4, 2015

[SPARK-10003] Improve readability of DAGScheduler

Andrew Or
authored and
kayousterhout
committed
[MINOR] Minor style fix in SparkR
shivaram
committed
MAINTENANCE: Automated closing of pull requests.
marmbrus
committed
[SPARK-10176] [SQL] Show partially analyzed plans when checkAnswer fails to analyze

cloud-fan
authored and
Andrew Or
committed
[SPARK-10450] [SQL] Minor improvements to readability / style / typos etc.
Andrew Or
committed
[SPARK-9669] [MESOS] Support PySpark on Mesos cluster mode.

tnachen
authored and
Andrew Or
committed
[SPARK-10454] [SPARK CORE] wait for empty event queue

robbins
authored and
Andrew Or
committed
[SPARK-10311] [STREAMING] Reload appId and attemptId when app starts with checkpoint file in cluster mode

XuTingjun
authored and
tdas
committed

Commits on Sep 5, 2015

[SPARK-10402] [DOCS] [ML] Add defaults to the scaladoc for params in ml/

holdenk
authored and
jkbradley
committed
[SPARK-9925] [SQL] [TESTS] Set SQLConf.SHUFFLE_PARTITIONS.key correctly for tests

yhuai
authored and
Andrew Or
committed
[HOTFIX] [SQL] Fixes compilation error

liancheng
authored and
rxin
committed
[SPARK-10440] [STREAMING] [DOCS] Update python API stuff in the programming guides and python docs

tdas
authored and
rxin
committed
[SPARK-10434] [SQL] Fixes Parquet schema of arrays that may contain null
liancheng
committed
[SPARK-10013] [ML] [JAVA] [TEST] remove java assert from java unit tests

holdenk
authored and
rxin
committed

Commits on Sep 7, 2015

[SPARK-9767] Remove ConnectionManager.
rxin
committed

Commits on Sep 8, 2015

[DOC] Added R to the list of languages with "high-level API" support in the…

Stephen Hopper
authored and
srowen
committed
Docs small fixes

jaceklaskowski
authored and
srowen
committed
[SPARK-9170] [SQL] Use OrcStructInspector to be case preserving when writing ORC files

viirya
authored and
liancheng
committed
[SPARK-10480] [ML] Fix ML.LinearRegressionModel.copy()

yanboliang
authored and
mengxr
committed
[SPARK-10316] [SQL] respect nondeterministic expressions in PhysicalOperation

cloud-fan
authored and
marmbrus
committed
[SPARK-10470] [ML] ml.IsotonicRegressionModel.copy should set parent

yanboliang
authored and
mengxr
committed
[SPARK-10441] [SQL] Save data correctly to json.

yhuai
authored and
marmbrus
committed
[SPARK-10468] [ MLLIB ] Verify schema before Dataframe select API call

Vinod K C
authored and
mengxr
committed
[SPARK-10492] [STREAMING] [DOCUMENTATION] Update Streaming documentation about rate limiting and backpressure
tdas
committed
[SPARK-10327] [SQL] Cache Table is not working while subquery has alias in its project list

chenghao-intel
authored and
marmbrus
committed
[HOTFIX] Fix build break caused by #8494
marmbrus
committed

Commits on Sep 9, 2015

[RELEASE] Add more contributors & only show names in release notes.
rxin
committed
[SPARK-10071] [STREAMING] Output a warning when writing QueueInputDStream and throw a better exception when reading QueueInputDStream

zsxwing
authored and
tdas
committed
[SPARK-9834] [MLLIB] implement weighted least squares via normal equation
mengxr
committed
[SPARK-10464] [MLLIB] Add WeibullGenerator for RandomDataGenerator

yanboliang
authored and
mengxr
committed
[SPARK-10373] [PYSPARK] move @since into pyspark from sql

Davies Liu
authored and
mengxr
committed
[SPARK-10094] Pyspark ML Feature transformers marked as experimental

noel-smith
authored and
mengxr
committed
[SPARK-9654] [ML] [PYSPARK] Add IndexToString to PySpark

holdenk
authored and
jkbradley
committed
[SPARK-10249] [ML] [DOC] Add Python Code Example to StopWordsRemover User Guide

hhbyyh
authored and
mengxr
committed
[SPARK-10227] fatal warnings with sbt on Scala 2.11

Luc Bourlier
authored and
srowen
committed
[SPARK-10117] [MLLIB] Implement SQL data source API for reading LIBSVM data

Lewuathe
authored and
mengxr
committed
[SPARK-10481] [YARN] SPARK_PREPEND_CLASSES make spark-yarn related jar could n…

zjffdu
authored and
Marcelo Vanzin
committed
[SPARK-10461] [SQL] make sure `input.primitive` is always variable name not code at `GenerateUnsafeProjection`

cloud-fan
authored and
davies
committed
[SPARK-9730] [SQL] Add Full Outer Join support for SortMergeJoin

viirya
authored and
davies
committed

Commits on Sep 10, 2015

[SPARK-9772] [PYSPARK] [ML] Add Python API for ml.feature.VectorSlicer

yanboliang
authored and
jkbradley
committed
[MINOR] [MLLIB] [ML] [DOC] fixed typo: label for negative result should be 0.0 (original: 1.0)

sparadiso
authored and
mengxr
committed
[SPARK-10497] [BUILD] [TRIVIAL] Handle both locations for JIRAError with python-jira

holdenk
authored and
srowen
committed
[SPARK-10065] [SQL] avoid the extra copy when generate unsafe array

cloud-fan
authored and
davies
committed
[SPARK-7142] [SQL] Minor enhancement to BooleanSimplification Optimizer rule

Yash Datta
authored and
marmbrus
committed
[SPARK-10301] [SPARK-10428] [SQL] Addresses comments of PR #8583 and #8509 for master

liancheng
authored and
davies
committed
[SPARK-10466] [SQL] UnsafeRow SerDe exception with data spill

chenghao-intel
authored and
Andrew Or
committed
[SPARK-10469] [DOC] Try and document the three options

holdenk
authored and
Andrew Or
committed
[SPARK-8167] Make tasks that fail from YARN preemption not fail job

mccheah
authored and
Andrew Or
committed
[SPARK-6350] [MESOS] Fine-grained mode scheduler respects mesosExecutor.cores

dragos
authored and
Andrew Or
committed
[SPARK-10514] [MESOS] waiting for min no of total cores acquired by Spark by implementing the sufficientResourcesRegistered method

SleepyThread
authored and
Andrew Or
committed
[SPARK-9990] [SQL] Create local hash join operator

zsxwing
authored and
Andrew Or
committed
[SPARK-10049] [SPARKR] Support collecting data of ArraryType in DataFrame.

Sun Rui
authored and
shivaram
committed
[SPARK-10443] [SQL] Refactor SortMergeOuterJoin to reduce duplication

Andrew Or
authored and
davies
committed
Add 1.5 to master branch EC2 scripts
shivaram
committed
[SPARK-7544] [SQL] [PySpark] pyspark.sql.types.Row implements __getitem__

yanboliang
authored and
davies
committed

Commits on Sep 11, 2015

[SPARK-9043] Serialize key, value and combiner classes in ShuffleDependency

massie
authored and
rxin
committed
[SPARK-10023] [ML] [PySpark] Unified DecisionTreeParams checkpointInterval between Scala and Python API.

yanboliang
authored and
mengxr
committed
[SPARK-10027] [ML] [PySpark] Add Python API missing methods for ml.feature

yanboliang
authored and
mengxr
committed
[SPARK-10472] [SQL] Fixes DataType.typeName for UDT
liancheng
committed
[SPARK-10556] Remove explicit Scala version for sbt project build files

ahirreddy
authored and
srowen
committed
[SPARK-10518] [DOCS] Update code examples in spark.ml user guide to use LIBSVM data source instead of MLUtils

y-shimizu
authored and
mengxr
committed
[SPARK-10026] [ML] [PySpark] Implement some common Params for regression in PySpark

yanboliang
authored and
mengxr
committed
[SPARK-9773] [ML] [PySpark] Add Python API for MultilayerPerceptronClassifier

yanboliang
authored and
mengxr
committed
[SPARK-10537] [ML] document LIBSVM source options in public API doc and some minor improvements
mengxr
committed
[MINOR] [MLLIB] [ML] [DOC] Minor doc fixes for StringIndexer and MetadataUtils

jkbradley
authored and
mengxr
committed
[SPARK-10540] [SQL] Ignore HadoopFsRelationTest's "test all data types" if it is too flaky
yhuai
committed
[SPARK-8530] [ML] add python API for MinMaxScaler

hhbyyh
authored and
jkbradley
committed
[SPARK-10546] Check partitionId's range in ExternalSorter#spill()

tedyu
authored and
srowen
committed
[PYTHON] Fixed typo in exception message

icaromedeiros
authored and
srowen
committed
[SPARK-10442] [SQL] fix string to boolean cast

cloud-fan
authored and
yhuai
committed
[SPARK-7142] [SQL] Minor enhancement to BooleanSimplification Optimizer rule. Incorporate review comments

Yash Datta
authored and
marmbrus
committed
[SPARK-9992] [SPARK-9994] [SPARK-9998] [SQL] Implement the local TopK, sample and intersect operators

zsxwing
authored and
Andrew Or
committed
[SPARK-9990] [SQL] Local hash join follow-ups
Andrew Or
committed
[SPARK-10564] ThreadingSuite: assertion failures in threads don't fail the test
Andrew Or
committed
[SPARK-9014] [SQL] Allow Python spark API to use built-in exponential operator

0x0FFF
authored and
davies
committed

Commits on Sep 13, 2015

[SPARK-10222] [GRAPHX] [DOCS] More thoroughly deprecate Bagel in favor of GraphX
srowen
committed

Commits on Sep 14, 2015