Skip to content

Clean up and simplify Spark configuration #299

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
pwendell wants to merge 32 commits into from
Closed

Clean up and simplify Spark configuration #299

Changes from 1 commit
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
ac2d65e
Change spark.local.dir -> SPARK_LOCAL_DIRS
pwendell Mar 31, 2014
0faa3b6
Stash of adding config options in submit script and YARN
pwendell Apr 1, 2014
6eaf7d0
executorJavaOpts
pwendell Apr 1, 2014
4982331
Remove SPARK_LIBRARY_PATH
pwendell Apr 1, 2014
1f75238
SPARK_JAVA_OPTS --> SPARK_MASTER_OPTS for master settings
pwendell Apr 1, 2014
84cc5e5
Small clean-up
pwendell Apr 1, 2014
5b0ba8e
Don't ship executor envs
pwendell Apr 2, 2014
7cc70e4
Clean up terminology inside of spark-env script
pwendell Apr 2, 2014
761ebcd
Library path and classpath for drivers
pwendell Apr 2, 2014
437aed1
Small fix
pwendell Apr 2, 2014
46555c1
Review feedback and import clean-ups
pwendell Apr 13, 2014
b72d183
Review feedback for spark env file
pwendell Apr 13, 2014
ace4ead
Responses to review feedback.
pwendell Apr 13, 2014
b08893b
Additional improvements.
pwendell Apr 13, 2014
afc9ed8
Cleaning up line limits and two compile errors.
pwendell Apr 14, 2014
4ee6f9d
Making YARN doc changes consistent
pwendell Apr 14, 2014
c2a2909
Test compile fixes
pwendell Apr 14, 2014
be42f35
Handle case where SPARK_HOME is not set
pwendell Apr 15, 2014
e83cd8f
Changes to allow re-use of test applications
pwendell Apr 15, 2014
308f1f6
Properly escape quotes and other clean-up for YARN
pwendell Apr 15, 2014
fda0301
Note
pwendell Apr 15, 2014
ffa00fe
Review feedback
pwendell Apr 18, 2014
a762901
Fixing test failures
pwendell Apr 18, 2014
d50c388
Merge remote-tracking branch 'apache/master' into config-cleanup
pwendell Apr 18, 2014
a56b125
Responses to Tom's review
pwendell Apr 18, 2014
af0adf7
Automatically add user jar
pwendell Apr 18, 2014
b16e6a2
Cleanup of spark-submit script and Scala quick start guide
pwendell Apr 20, 2014
af09e3e
Mention config file in docs and clean-up docs
pwendell Apr 21, 2014
0086939
Minor style fixes
pwendell Apr 21, 2014
b4b496c
spark-defaults.properties -> spark-defaults.conf
pwendell Apr 21, 2014
a006464
Moving properties file template.
pwendell Apr 21, 2014
127f301
Improvements to testing
pwendell Apr 21, 2014
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Moving properties file template.
  • Loading branch information
@pwendell
pwendell committed Apr 21, 2014
commit a0064640394a97a114cdcca4de00666c802605bb
0 conf/spark-defaults.properties.template → conf/spark-defaults.conf.template
View file
Open in desktop
File renamed without changes.