[STREAMING]There is a dependent package conflict#8738

Closed

272029252 wants to merge 511 commits intomasterfrom

Commits on Aug 15, 2015

[SPARK-9725] [SQL] fix serialization of UTF8String across different JVM

Davies Liu
authored and
davies
committed
[SPARK-9980] [BUILD] Fix SBT publishLocal error due to invalid characters in doc

hvanhovell
authored and
srowen
committed
[SPARK-9955] [SQL] correct error message for aggregate

cloud-fan
authored and
marmbrus
committed

Commits on Aug 17, 2015

[SPARK-9871] [SPARKR] Add expression functions into SparkR which have a variable parameter

yu-iskw
authored and
shivaram
committed
[SPARK-9959] [MLLIB] Association Rules Java Compatibility

Feynman Liang
authored and
mengxr
committed
[SPARK-7837] [SQL] Avoids double closing output writers when commitTask() fails
liancheng
committed
[SPARK-9950] [SQL] Wrong Analysis Error for grouping/aggregating on struct fields

cloud-fan
authored and
marmbrus
committed
[SPARK-10036] [SQL] Load JDBC driver in DataFrameReader.jdbc and DataFrameWriter.jdbc

zsxwing
authored and
marmbrus
committed
[SPARK-9526] [SQL] Utilize randomized tests to reveal potential bugs in sql expressions

yjshen
authored and
JoshRosen
committed
[SPARK-9592] [SQL] Fix Last function implemented based on AggregateExpression1.

yhuai
authored and
marmbrus
committed
[SPARK-10068] [MLLIB] Adds links to MLlib types, algos, utilities listing

Feynman Liang
authored and
mengxr
committed
[SPARK-8920] [MLLIB] Add @since tags to mllib.linalg

Sameer Abhyankar
authored and
mengxr
committed

Commits on Aug 18, 2015

[SPARK-9974] [BUILD] [SQL] Makes sure com.twitter:parquet-hadoop-bundle:1.6.0 is in SBT assembly jar

liancheng
authored and
rxin
committed
[SPARK-9768] [PYSPARK] [ML] Add Python API and user guide for ml.feature.ElementwiseProduct

yanboliang
authored and
jkbradley
committed
SPARK-8916 [Documentation, MLlib] Add @since tags to mllib.regression

prayagchandran
authored and
DB Tsai
committed
[SPARK-9898] [MLLIB] Prefix Span user guide

Feynman Liang
authored and
mengxr
committed
[SPARK-7707] User guide and example code for KernelDensity

sryza
authored and
mengxr
committed
[SPARK-9902] [MLLIB] Add Java and Python examples to user guide for 1-sample KS test

jose.cambronero
authored and
mengxr
committed
[SPARK-10059] [YARN] Explicitly add JSP dependencies for tests.
Marcelo Vanzin
committed
[SPARK-7808] [ML] add package doc for ml.feature
mengxr
committed
[MINOR] Format the comment of `translate` at `functions.scala`

yu-iskw
authored and
rxin
committed
[SPARK-10038] [SQL] fix bug in generated unsafe projection when there is binary in ArrayData

Davies Liu
authored and
rxin
committed
[SPARK-10076] [ML] make MultilayerPerceptronClassifier layers and weights public

yanboliang
authored and
mengxr
committed
[MINOR] fix the comments in IndexShuffleBlockResolver

CodingCat
authored and
srowen
committed
[SPARK-8118] [SQL] Redirects Parquet JUL logger via SLF4J
liancheng
committed
[SPARK-10007] [SPARKR] Update `NAMESPACE` file in SparkR for simple parameters functions

yu-iskw
authored and
shivaram
committed
[SPARK-9028] [ML] Add CountVectorizer as an estimator to generate CountVectorizerModel

hhbyyh
authored and
jkbradley
committed
[SPARK-9900] [MLLIB] User guide for Association Rules

Feynman Liang
authored and
mengxr
committed
[SPARK-10029] [MLLIB] [DOC] Add Python examples for mllib IsotonicRegression user guide

yanboliang
authored and
mengxr
committed
[SPARK-10032] [PYSPARK] [DOC] Add Python example for mllib LDAModel user guide

yanboliang
authored and
mengxr
committed
[SPARK-10085] [MLLIB] [DOCS] removed unnecessary numpy array import

stared
authored and
mengxr
committed
[SPARK-9574] [STREAMING] Remove unnecessary contents of spark-streaming-XXX-assembly jars

zsxwing
authored and
tdas
committed
[SPARK-10080] [SQL] Fix binary incompatibility for $ column interpolation
marmbrus
committed
[SPARK-10089] [SQL] Add missing golden files.

Marcelo Vanzin
authored and
marmbrus
committed
[SPARK-10088] [SQL] Add support for "stored as avro" in HiveQL parser.

Marcelo Vanzin
authored and
marmbrus
committed
[SPARK-8924] [MLLIB, DOCUMENTATION] Added @since tags to mllib.tree

BryanCutler
authored and
mengxr
committed
[SPARK-10012] [ML] Missing test case for Params#arrayLengthGt

Lewuathe
authored and
jkbradley
committed

Commits on Aug 19, 2015

[SPARK-10098] [STREAMING] [TEST] Cleanup active context after test in FailureSuite
tdas
committed
[SPARK-8473] [SPARK-9889] [ML] User guide and example code for DCT

Feynman Liang
authored and
jkbradley
committed
Bump SparkR version string to 1.5.0

falaki
authored and
shivaram
committed
[SPARKR] [MINOR] Get rid of a long line warning

yu-iskw
authored and
shivaram
committed
[SPARK-10072] [STREAMING] BlockGenerator can deadlock when the queue of generate blocks fills up to capacity
tdas
committed
[SPARK-10102] [STREAMING] Fix a race condition that startReceiver may happen before setting trackerState to Started

zsxwing
authored and
tdas
committed
[SPARK-9939] [SQL] Resorts to Java process API in CliSuite, HiveSparkSubmitSuite and HiveThriftServer2 test suites
liancheng
committed
[SPARK-10075] [SPARKR] Add `when` expressino function in SparkR

yu-iskw
authored and
shivaram
committed
[SPARK-10095] [SQL] use public API of BigInteger

Davies Liu
authored and
davies
committed
[SPARK-10093] [SPARK-10096] [SQL] Avoid transformation on executors & fix UDFs on complex types
rxin
committed
[SPARK-9705] [DOC] fix docs about Python version

Davies Liu
authored and
rxin
committed
[SPARK-9508] GraphX Pregel docs update with new Pregel code

avulanov
authored and
rxin
committed
[SPARK-9952] Fix N^2 loop when DAGScheduler.getPreferredLocsInternal accesses cacheLocs

JoshRosen
authored and
rxin
committed
[SPARK-9967] [SPARK-10099] [STREAMING] Renamed conf spark.streaming.backpressure.{enable-->enabled} and fixed deprecated annotations
tdas
committed
Fix Broken Link

bllchmbrs
authored and
rxin
committed
[SPARK-10070] [DOCS] Remove Guava dependencies in user guides
srowen
committed
[DOCS] [SQL] [PYSPARK] Fix typo in ntile function

moutai
authored and
srowen
committed
[SPARK-9977] [DOCS] Update documentation for StringIndexer

Lewuathe
authored and
srowen
committed
[SPARK-8949] Print warnings when using preferred locations feature

darkjh
authored and
srowen
committed
[SPARK-10060] [ML] [DOC] spark.ml DecisionTree user guide

jkbradley
authored and
mengxr
committed
[SPARK-10084] [MLLIB] [DOC] Add Python example for mllib FP-growth user guide

yanboliang
authored and
mengxr
committed
[SPARK-9856] [SPARKR] Add expression functions into SparkR whose params are complicated

yu-iskw
authored and
shivaram
committed
[SPARK-10097] Adds `shouldMaximize` flag to `ml.evaluation.Evaluator`

Feynman Liang
authored and
jkbradley
committed
[SPARK-10106] [SPARKR] Add `ifelse` Column function to SparkR

yu-iskw
authored and
shivaram
committed
[SPARK-8918] [MLLIB] [DOC] Add @since tags to mllib.clustering
mengxr
committed
[SPARK-10107] [SQL] fix NPE in format_number

Davies Liu
authored and
rxin
committed
[SPARK-10087] [CORE] [BRANCH-1.5] Disable spark.shuffle.reduceLocality.enabled by default.

yhuai
authored and
rxin
committed
[SPARK-10073] [SQL] Python withColumn should replace the old column

Davies Liu
authored and
marmbrus
committed
[SPARK-9627] [SQL] Stops using Scala runtime reflection in DictionaryEncoding

liancheng
authored and
marmbrus
committed
[SPARK-10090] [SQL] fix decimal scale of division

Davies Liu
authored and
marmbrus
committed
[SPARK-9899] [SQL] Disables customized output committer when speculation is on

liancheng
authored and
marmbrus
committed
[SPARK-10083] [SQL] CaseWhen should support type coercion of DecimalType and FractionalType

adrian-wang
authored and
marmbrus
committed
[SPARK-10119] [CORE] Fix isDynamicAllocationEnabled when config is expliticly disabled.
Marcelo Vanzin
committed
[SPARK-6489] [SQL] add column pruning for Generate

cloud-fan
authored and
marmbrus
committed
[SPARK-9895] User Guide for RFormula Feature Transformer

ericl
authored and
mengxr
committed

Commits on Aug 20, 2015

Commits on Aug 21, 2015

Commits on Aug 22, 2015

Update programming-guide.md

yosssi
authored and
rxin
committed

Commits on Aug 23, 2015

Update streaming-programming-guide.md

yosssi
authored and
srowen
committed

Commits on Aug 24, 2015

Commits on Aug 25, 2015

HOTFIX: Adding missing 1.4.1 ec2 version
pwendell
committed
[SPARK-10165] [SQL] Await child resolution in ResolveFunctions
marmbrus
committed
[SPARK-10118] [SPARKR] [DOCS] Improve SparkR API docs for 1.5 release

yu-iskw
authored and
shivaram
committed
[SQL] [MINOR] [DOC] Clarify docs for inferring DataFrame from RDD of Products

Feynman Liang
authored and
rxin
committed
[SPARK-10121] [SQL] Thrift server always use the latest class loader provided by the conf of executionHive's state

yhuai
authored and
liancheng
committed
[SPARK-10178] [SQL] HiveComparisionTest should print out dependent tables

marmbrus
authored and
rxin
committed
[SPARK-9786] [STREAMING] [KAFKA] fix backpressure so it works with defa…

koeninger
authored and
tdas
committed
[SPARK-10137] [STREAMING] Avoid to restart receivers if scheduleReceivers returns balanced results

zsxwing
authored and
tdas
committed
[SPARK-10196] [SQL] Correctly saving decimals in internal rows to JSON.

yhuai
authored and
davies
committed
[SPARK-10136] [SQL] A more robust fix for SPARK-10136
liancheng
committed
[SPARK-9293] [SPARK-9813] Analysis should check that set operations are only performed on tables with equal numbers of columns

JoshRosen
authored and
marmbrus
committed
[SPARK-10214] [SPARKR] [DOCS] Improve SparkR Column, DataFrame API docs

yu-iskw
authored and
shivaram
committed
[SPARK-6196] [BUILD] Remove MapR profiles in favor of hadoop-provided
srowen
committed
[SPARK-10210] [STREAMING] Filter out non-existent blocks before creating BlockRDD
tdas
committed
[SPARK-10177] [SQL] fix reading Timestamp in parquet from Hive

Davies Liu
authored and
liancheng
committed
[SPARK-10195] [SQL] Data sources Filter should not expose internal types

JoshRosen
authored and
rxin
committed
[SPARK-10197] [SQL] Add null check in wrapperFor (inside HiveInspectors).

yhuai
authored and
liancheng
committed
[DOC] add missing parameters in SparkContext.scala for scala doc

liyezhang556520
authored and
srowen
committed
Fixed a typo in DAGScheduler.

zzvara
authored and
srowen
committed
[SPARK-10198] [SQL] Turn off partition verification by default
marmbrus
committed
[SPARK-8531] [ML] Update ML user guide for MinMaxScaler

hhbyyh
authored and
jkbradley
committed
[SPARK-10230] [MLLIB] Rename optimizeAlpha to optimizeDocConcentration

Feynman Liang
authored and
jkbradley
committed
[SPARK-10231] [MLLIB] update @Since annotation for mllib.classification

mengxr
authored and
DB Tsai
committed
[SPARK-9800] Adds docs for GradientDescent$.runMiniBatchSGD alias

Feynman Liang
authored and
jkbradley
committed
[SPARK-10237] [MLLIB] update since versions in mllib.fpm
mengxr
committed
[SPARK-9797] [MLLIB] [DOC] StreamingLinearRegressionWithSGD.setConvergenceTol default value

Feynman Liang
authored and
jkbradley
committed
[SPARK-10239] [SPARK-10244] [MLLIB] update since versions in mllib.pmml and mllib.util

mengxr
authored and
DB Tsai
committed
[SPARK-10245] [SQL] Fix decimal literals with precision < scale

Davies Liu
authored and
yhuai
committed
[SPARK-10215] [SQL] Fix precision of division (follow the rule in Hive)

Davies Liu
authored and
yhuai
committed
Preparing Spark release v1.5.0-rc2
pwendell
committed
Preparing development version 1.5.1-SNAPSHOT
pwendell
committed

Commits on Aug 26, 2015

[SPARK-9888] [MLLIB] User guide for new LDA features

Feynman Liang
authored and
jkbradley
committed
[SPARK-10233] [MLLIB] update since version in mllib.evaluation
mengxr
committed
[SPARK-10238] [MLLIB] update since versions in mllib.linalg

mengxr
authored and
DB Tsai
committed
[SPARK-10240] [SPARK-10242] [MLLIB] update since versions in mlilb.random and mllib.stat
mengxr
committed
[SPARK-10234] [MLLIB] update since version in mllib.clustering
mengxr
committed
[SPARK-10243] [MLLIB] update since versions in mllib.tree
mengxr
committed
[SPARK-10235] [MLLIB] update since versions in mllib.regression

mengxr
authored and
DB Tsai
committed
[SPARK-10236] [MLLIB] update since versions in mllib.feature

mengxr
authored and
DB Tsai
committed
[SPARK-9316] [SPARKR] Add support for filtering using `[` (synonym for filter / select)

felixcheung
authored and
shivaram
committed
[SPARK-9665] [MLLIB] audit MLlib API annotations

mengxr
authored and
jkbradley
committed
[SPARK-10241] [MLLIB] update since versions in mllib.recommendation
mengxr
committed
[SPARK-10305] [SQL] fix create DataFrame from Python class

Davies Liu
authored and
davies
committed

Commits on Aug 27, 2015

[SPARK-10308] [SPARKR] Add %in% to the exported namespace
shivaram
committed
[SPARK-9424] [SQL] Parquet programming guide updates for 1.5

liancheng
authored and
marmbrus
committed
[SPARK-10219] [SPARKR] Fix varargsToEnv and add test case
shivaram
committed
[DOCS] [STREAMING] [KAFKA] Fix typo in exactly once semantics

moutai
authored and
srowen
committed
[SPARK-9148] [SPARK-10252] [SQL] Update SQL Programming Guide
marmbrus
committed
[SPARK-10315] remove document on spark.akka.failure-detector.threshold

CodingCat
authored and
srowen
committed
[SPARK-9901] User guide for RowMatrix Tall-and-skinny QR

hhbyyh
authored and
mengxr
committed
[SPARK-9906] [ML] User guide for LogisticRegressionSummary

MechCoder
authored and
jkbradley
committed
[SPARK-9680] [MLLIB] [DOC] StopWordsRemovers user guide and Java compatibility test

Feynman Liang
authored and
mengxr
committed
[SPARK-10287] [SQL] Fixes JSONRelation refreshing on read path
yhuai
committed
[SPARK-10321] sizeInBytes in HadoopFsRelation

Davies Liu
authored and
marmbrus
committed

Commits on Aug 28, 2015

[SPARK-9911] [DOC] [ML] Update Userguide for Evaluator

MechCoder
authored and
mengxr
committed
[SPARK-9905] [ML] [DOC] Adds LinearRegressionSummary user guide

Feynman Liang
authored and
mengxr
committed
[SPARK-SQL] [MINOR] Fixes some typos in HiveContext

liancheng
authored and
rxin
committed
[SPARK-10188] [PYSPARK] Pyspark CrossValidator with RMSE selects incorrect model

noel-smith
authored and
jkbradley
committed
[SPARK-10328] [SPARKR] Fix generic for na.omit
shivaram
committed
[SPARK-10035] [SQL] Parquet filters does not process EqualNullSafe filter.

HyukjinKwon
authored and
liancheng
committed
[SPARK-10295] [CORE] Dynamic allocation in Mesos does not release when RDDs are cached
srowen
committed
Fix DynamodDB/DynamoDB typo in Kinesis Integration doc

yosssi
authored and
srowen
committed
typo in comment

dharmeshkakadia
authored and
srowen
committed
[SPARK-9890] [DOC] [ML] User guide for CountVectorizer

hhbyyh
authored and
mengxr
committed
[SPARK-8952] [SPARKR] - Wrap normalizePath calls with suppressWarnings

lresende
authored and
shivaram
committed
[SPARK-10325] Override hashCode() for public Row

JoshRosen
authored and
marmbrus
committed
[SPARK-10336][example] fix not being able to set intercept in LR example

Shuo Xiang
authored and
DB Tsai
committed
[SPARK-9671] [MLLIB] re-org user guide and add migration guide
mengxr
committed
[SPARK-10323] [SQL] fix nullability of In/InSet/ArrayContain

Davies Liu
authored and
davies
committed
[SPARK-10326] [YARN] Fix app submission on windows.

Marcelo Vanzin
authored and
rxin
committed

Commits on Aug 29, 2015

[SPARK-9803] [SPARKR] Add subset and transform + tests

felixcheung
authored and
shivaram
committed
[SPARK-9910] [ML] User guide for train validation split

zapletal-martin
authored and
mengxr
committed
[SPARK-10350] [DOC] [SQL] Removed duplicated option description from SQL guide

witgo
authored and
marmbrus
committed
[SPARK-10344] [SQL] Add tests for extraStrategies

marmbrus
authored and
yhuai
committed
[SPARK-10226] [SQL] Fix exclamation mark issue in SparkSQL

small-wang
authored and
marmbrus
committed
[SPARK-10330] Use SparkHadoopUtil TaskAttemptContext reflection methods in more places

JoshRosen
authored and
marmbrus
committed
[SPARK-10339] [SPARK-10334] [SPARK-10301] [SQL] Partitioned table scan can OOM driver and throw a better error message when users need to enable parquet schema merging

yhuai
authored and
marmbrus
committed

Commits on Aug 30, 2015

Commits on Sep 1, 2015

Commits on Sep 3, 2015

[SPARK-10422] [SQL] String column in InMemoryColumnarCache needs to override clone method

yhuai
authored and
davies
committed
[SPARK-10379] preserve first page in UnsafeShuffleExternalSorter

Davies Liu
authored and
Andrew Or
committed
[SPARK-10411] [SQL] Move visualization above explain output and hide explain by default

zsxwing
authored and
Andrew Or
committed
[SPARK-10332] [CORE] Fix yarn spark executor validation

holdenk
authored and
srowen
committed
[SPARK-9869] [STREAMING] Wait for all event notifications before asserting results

robbins
authored and
Andrew Or
committed
[SPARK-10431] [CORE] Fix intermittent test failure. Wait for event queue to be clear

robbins
authored and
Andrew Or
committed

Commits on Sep 4, 2015

[SPARK-10454] [SPARK CORE] wait for empty event queue

robbins
authored and
Andrew Or
committed
[SPARK-10311] [STREAMING] Reload appId and attemptId when app starts with checkpoint file in cluster mode

XuTingjun
authored and
tdas
committed

Commits on Sep 5, 2015

Commits on Sep 8, 2015

Commits on Sep 9, 2015

[SPARK-10301] [SPARK-10428] [SQL] [BRANCH-1.5] Fixes schema merging for nested structs

liancheng
authored and
yhuai
committed
[SPARK-10071] [STREAMING] Output a warning when writing QueueInputDStream and throw a better exception when reading QueueInputDStream

zsxwing
authored and
tdas
committed
[SPARK-7736] [CORE] [YARN] Make pyspark fail YARN app on failure.
Marcelo Vanzin
committed
[SPARK-7736] [CORE] Fix a race introduced in PythonRunner.
Marcelo Vanzin
committed

Commits on Sep 10, 2015

[MINOR] [MLLIB] [ML] [DOC] fixed typo: label for negative result should be 0.0 (original: 1.0)

sparadiso
authored and
mengxr
committed
[SPARK-10466] [SQL] UnsafeRow SerDe exception with data spill

chenghao-intel
authored and
Andrew Or
committed
[SPARK-10469] [DOC] Try and document the three options

holdenk
authored and
Andrew Or
committed
[SPARK-6350] [MESOS] Fine-grained mode scheduler respects mesosExecutor.cores

dragos
authored and
Andrew Or
committed
Revert "[SPARK-6350] [MESOS] Fine-grained mode scheduler respects mesosExecutor.cores"
Andrew Or
committed

Commits on Sep 11, 2015

[SPARK-10556] Remove explicit Scala version for sbt project build files

ahirreddy
authored and
srowen
committed
[SPARK-10540] [SQL] Ignore HadoopFsRelationTest's "test all data types" if it is too flaky
yhuai
committed
[SPARK-9924] [WEB UI] Don't schedule checkForLogs while some of them …

Rohit Agarwal
authored and
Marcelo Vanzin
committed
[SPARK-10564] ThreadingSuite: assertion failures in threads don't fail the test
Andrew Or
committed

Commits on Sep 12, 2015

[SPARK-10566] [CORE] SnappyCompressionCodec init exception handling masks important error information

dimfeld
authored and
srowen
committed
[SPARK-10554] [CORE] Fix NPE with ShutdownHook

Nithin Asokan
authored and
srowen
committed

Commits on Sep 13, 2015

[SPARK-6350] [MESOS] [BACKPORT] Fine-grained mode scheduler respects

dragos
authored and
srowen
committed

Commits on Sep 14, 2015

[SPARK-10584] [DOC] [SQL] Documentation about spark.sql.hive.metastore.version is wrong.

sarutak
authored and
yhuai
committed
[SPARK-10573] [ML] IndexToString output schema should be StringType

Nick Pritchard
authored and
mengxr
committed
[SPARK-10522] [SQL] Nanoseconds of Timestamp in Parquet should be positive

Davies Liu
authored and
davies
committed
[SPARK-10549] scala 2.11 spark on yarn with security - Repl doesn't work

Tom Graves
authored and
Andrew Or
committed
[SPARK-10543] [CORE] Peak Execution Memory Quantile should be Per-task Basis

saurfang
authored and
Andrew Or
committed
[SPARK-10564] ThreadingSuite: assertion failures in threads don't fail the test (round 2)
Andrew Or
committed

Commits on Sep 15, 2015

[SPARK-10542] [PYSPARK] fix serialize namedtuple

Davies Liu
authored and
davies
committed
Small fixes to docs

Jacek Laskowski
authored and
rxin
committed