Skip to content

[WIP] [SQL] Experiment with speculatively running without reset() in HiveCompatibilitySuite #6663

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
JoshRosen wants to merge 26 commits into from
Closed

[WIP] [SQL] Experiment with speculatively running without reset() in HiveCompatibilitySuite #6663

Changes from 1 commit
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
00bcf8a
Avoid IO operations on empty files in BlockObjectWriter.
JoshRosen Apr 21, 2015
8fd89b4
Do not create empty files at all.
JoshRosen Apr 21, 2015
0db87c3
Reduce scope of FileOutputStream in ExternalSorter
JoshRosen Apr 21, 2015
7e2340d
Revert "Reduce scope of FileOutputStream in ExternalSorter"
JoshRosen Apr 22, 2015
8113eac
Hacky WIP towards speculatively running w/o reset(), then retrying wi…
JoshRosen Jun 5, 2015
417f50e
Hackily comment out most of dev/run-tests to speed up Jenkins iteration.
JoshRosen Jun 5, 2015
f90dc94
Don't pass configuration to ObjectWritable in SerializableWritable
JoshRosen Jun 5, 2015
480d20a
Broadcast configuration in hiveWriterContainers (WIP hack)
JoshRosen Jun 5, 2015
55041d2
Use local[*] instead of local[2]
JoshRosen Jun 5, 2015
57c2cb4
try in-memory Derby
JoshRosen Jun 5, 2015
9db0abc
Avoid writing empty files in BypassMergeSortShuffleWriter
JoshRosen Jun 5, 2015
9e116d1
Rework SPARK-7041 for BypassMergeSort split
JoshRosen Jun 5, 2015
3fe16e8
Revert "Broadcast configuration in hiveWriterContainers (WIP hack)"
JoshRosen Jun 5, 2015
54cd5ce
Merge remote-tracking branch 'origin/master' into file-handle-optimiz…
JoshRosen Jun 5, 2015
5c777cf
Rework SPARK-7041 for BypassMergeSort split
JoshRosen Jun 5, 2015
5ac11d1
Revert "Use local[*] instead of local[2]"
JoshRosen Jun 5, 2015
895de59
Revert "try in-memory Derby"
JoshRosen Jun 5, 2015
bf30fee
Revert "Don't pass configuration to ObjectWritable in SerializableWri…
JoshRosen Jun 5, 2015
b1e3f82
Revert "Hacky WIP towards speculatively running w/o reset(), then ret…
JoshRosen Jun 5, 2015
5113370
Revert "Rework SPARK-7041 for BypassMergeSort split"
JoshRosen Jun 5, 2015
fac08d5
SPARK-8135. In SerializableWritable, don't load defaults when instant…
sryza Jun 5, 2015
c7caa5c
Merge remote-tracking branch 'origin/master' into file-handle-optimiz…
JoshRosen Jun 6, 2015
8c1e1ff
Merge branch 'file-handle-optimizations' into hive-compat-suite-speedup
JoshRosen Jun 8, 2015
2b500b9
Only run unsafe tests (testing a jenkins job)
JoshRosen Jun 12, 2015
fbd3d03
Add log4j test properties to unsafe project.
JoshRosen Jun 12, 2015
46dd005
Try only testing bagel instead (since I don't think Maven logs to uni…
JoshRosen Jun 12, 2015
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
try in-memory Derby
  • Loading branch information
@JoshRosen
JoshRosen committed Jun 5, 2015
commit 57c2cb444070a5dc9cd01f7f9c0004e8c24e00dc
2 changes: 1 addition & 1 deletion sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveContext.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -536,7 +536,7 @@ private[hive] object HiveContext {
}
}
propMap.put("javax.jdo.option.ConnectionURL",
s"jdbc:derby:;databaseName=$localMetastore;create=true")
s"jdbc:derby:memory:myDB;create=true")
propMap.put("datanucleus.rdbms.datastoreAdapterClassName",
"org.datanucleus.store.rdbms.adapter.DerbyAdapter")
propMap.toMap
Expand Down