[SPARK-6954] [YARN] ExecutorAllocationManager can end up requesting a ne...

[sryza]

gative number of executors

[sryza]

@andrewor14

[AmplabJenkins]

Merged build triggered.

[AmplabJenkins]

Merged build started.

[SparkQA]

Test build #31645 has started for PR 5856 at commit 1cb517a.

• This patch merges cleanly.

[SparkQA]

Test build #31645 has finished for PR 5856 at commit 1cb517a.

• This patch fails Spark unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[AmplabJenkins]

Merged build finished. Test FAILed.

[AmplabJenkins]

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/31645/
Test FAILed.

[andrewor14]

retest this please

[AmplabJenkins]

Merged build triggered.

[AmplabJenkins]

Merged build started.

[SparkQA]

Test build #31648 has started for PR 5856 at commit 1cb517a.

• This patch merges cleanly.

[SparkQA]

Test build #31648 has finished for PR 5856 at commit 1cb517a.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[AmplabJenkins]

Merged build finished. Test PASSed.

[AmplabJenkins]

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/31648/
Test PASSed.

[andrewor14]

I'm merging this into 1.3

[andrewor14]

Hm, merge script is acting up a little... for some reason I can't merge this.

[srowen]

Yeah, I get...

    raise CalledProcessError(retcode, cmd, output=output)
subprocess.CalledProcessError: Command '['git', 'commit', '--author="Sandy Ryza <[email protected]>"', '-m', u'[SPARK-6954] [YARN] ExecutorAllocationManager can end up requesting a ne...', '-m', u'...gative n...\r\n\r\n...umber of executors\r\n\r\nAuthor: Sandy Ryza <sandycloudera.com>\r\n\r\nCloses #5704 from sryza/sandy-spark-6954 and squashes the following commits:\r\n\r\nb7890fb [Sandy Ryza] Avoid ramping up to an existing number of executors\r\n6eb516a [Sandy Ryza] SPARK-6954. ExecutorAllocationManager can end up requesting a negative number of executors', '-m', 'Author: Sandy Ryza <[email protected]>', '-m', u'Closes #5856 from sryza/sandy-backport-6954 and squashes the following commits:', '-m', '1cb517a [Sandy Ryza] [SPARK-6954] [YARN] ExecutorAllocationManager can end up requesting a negative n...']' returned non-zero exit status 1

Not sure why git fails. Maybe I can try executing this command by hand, carefully.

[srowen]

My guess is it's something to do with the text of the commit message, since it comes from a copy and paste of a commit message. Maybe something doesn't quite get escaped or confuses git.

I tried this just after failure but didn't work for other reasons (untracked files in the working dir; guess I'd have to do something else at this point in the script to make it work but I'm not sure what):

git commit --author="Sandy Ryza <[email protected]>" -m "[SPARK-6954] [YARN] ExecutorAllocationManager can end up requesting a negative number of executors." -m "Author: Sandy Ryza <[email protected]>" -m "Closes #5856 from sryza/sandy-backport-6954 and squashes the following commits:" -m "1cb517a [Sandy Ryza] [SPARK-6954] [YARN] ExecutorAllocationManager can end up requesting a negative n..."

@sryza to test my theory are you able to simplify the description to not include the old commit message stuff? maybe that just makes it work.

[andrewor14]

Interesting. That might actually be the case. @sryza could you update the description?

[sryza]

@andrewor14 @srowen updated it.

[andrewor14]

Weird, it still doesn't work. @sryza @srowen do you want to give it a try?

[srowen]

OK, I'm confused and/or dumb, but this failed because it was already merged, by me:
98ac39d
(i.e. git says "nothing is different" and exits with status 1)
I don't recall how/why I merged that when Andrew merged the other bit, when it might have looked like it wasn't auto-mergeable for 1.3. Anyway, this can be closed.

[andrewor14]

Haha wow, good to know. Let's close this then.

[andrewor14]

Hey @sryza can you close this PR? It's already merged.

[sryza]

Oops sorry for the delay