Skip to content

Commit 71fc113

Browse files
committed
Merge pull request apache#349 from CodingCat/support-worker_dir
add the comments about SPARK_WORKER_DIR this env variable seems to be forgotten in many cases we need to set this variable, e.g. in EC2, we have to move the large application log files from the EBS to the ephemeral storage
2 parents 15d9534 + 3633172 commit 71fc113

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

conf/spark-env.sh.template

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -18,4 +18,4 @@
1818
# - SPARK_WORKER_MEMORY, to set how much memory to use (e.g. 1000m, 2g)
1919
# - SPARK_WORKER_PORT / SPARK_WORKER_WEBUI_PORT
2020
# - SPARK_WORKER_INSTANCES, to set the number of worker processes per node
21-
21+
# - SPARK_WORKER_DIR, to set the working directory of worker processes

0 commit comments

Comments
 (0)