Skip to content

A merge of latest changes since creation #1

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
yzhou2001 merged 13 commits into from
Aug 14, 2014
Merged

A merge of latest changes since creation #1

Changes from 1 commit
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
9497b12
[SPARK-3006] Failed to execute spark-shell in Windows OS
tsudukim Aug 14, 2014
e424565
[Docs] Add missing <code> tags (minor)
andrewor14 Aug 14, 2014
69a57a1
[SPARK-2995][MLLIB] add ALS.setIntermediateRDDStorageLevel
mengxr Aug 14, 2014
d069c5d
[SPARK-3029] Disable local execution of Spark jobs by default
aarondav Aug 14, 2014
6b8de0e
SPARK-2893: Do not swallow Exceptions when running a custom kryo regi…
GrahamDennis Aug 14, 2014
078f3fb
[SPARK-3011][SQL] _temporary directory should be filtered out by sqlC…
josephsu Aug 14, 2014
add75d4
[SPARK-2927][SQL] Add a conf to configure if we always read Binary co…
yhuai Aug 14, 2014
fde692b
[SQL] Python JsonRDD UTF8 Encoding Fix
ahirreddy Aug 14, 2014
267fdff
[SPARK-2925] [sql]fix spark-sql and start-thriftserver shell bugs whe…
scwf Aug 14, 2014
eaeb0f7
Minor cleanup of metrics.Source
rxin Aug 14, 2014
9622106
[SPARK-2979][MLlib] Improve the convergence rate by minimizing the co…
Aug 14, 2014
a7f8a4f
Revert [SPARK-3011][SQL] _temporary directory should be filtered out…
marmbrus Aug 14, 2014
a75bc7a
SPARK-3009: Reverted readObject method in ApplicationInfo so that App…
jacek-lewandowski Aug 14, 2014
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Revert [SPARK-3011][SQL] _temporary directory should be filtered out … 
…by sqlContext.parquetFile

Reverts apache#1924 due to build failures with hadoop 0.23.

Author: Michael Armbrust <[email protected]>

Closes apache#1949 from marmbrus/revert1924 and squashes the following commits:

6bff940 [Michael Armbrust] Revert "[SPARK-3011][SQL] _temporary directory should be filtered out by sqlContext.parquetFile"
  • Loading branch information
@marmbrus
marmbrus committed Aug 14, 2014
commit a7f8a4f5ee757450ce8d4028021441435081cf53
3 changes: 1 addition & 2 deletions sql/core/src/main/scala/org/apache/spark/sql/parquet/ParquetTypes.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -378,8 +378,7 @@ private[parquet] object ParquetTypesConverter extends Logging {

val children = fs.listStatus(path).filterNot { status =>
val name = status.getPath.getName
name(0) == '.' || name == FileOutputCommitter.SUCCEEDED_FILE_NAME ||
name == FileOutputCommitter.TEMP_DIR_NAME
name(0) == '.' || name == FileOutputCommitter.SUCCEEDED_FILE_NAME
}

// NOTE (lian): Parquet "_metadata" file can be very slow if the file consists of lots of row
Expand Down