Skip to content

[WIP][SPARK-7018][Build]: Refactor dev/run-tests-jenkins into Python #7401

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
brennonyork wants to merge 34 commits into from
Closed

[WIP][SPARK-7018][Build]: Refactor dev/run-tests-jenkins into Python #7401

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
31b51de
initial cut of refactored run-tests-jenkins script into python
Jul 13, 2015
f2a1dc6
fixed pep8 issues
Jul 14, 2015
d202c42
fixed list bug when finding log files
Jul 14, 2015
3ae1d49
correctly escape newlines for long strings
Jul 14, 2015
2a3d67d
fixed numerous bugs, updated to use urllib2 over curl, removed 'find'…
Jul 15, 2015
d83c535
fixed typos, style, and linting issues
Jul 15, 2015
3acac90
fixed indenting with list
Jul 15, 2015
395928d
fixed typo
Jul 15, 2015
30f90bc
fixed merge conflicts
Jul 15, 2015
4765dc8
added variables to remove indents, fixed ' : ' => ': ' in maps
Jul 15, 2015
e90fdeb
moved if statement for urllib2 into the try block
Jul 15, 2015
64c13c1
updated token passing for github oauth
Jul 15, 2015
85aaf76
added subprocess_check_call() as backport from python 2.7, moved subp…
Jul 15, 2015
f756098
removed unnecessary escapes
Jul 15, 2015
39f6a1f
changed test process to only return the returncode
Jul 15, 2015
7b11665
fixed serialization error between int and string, created an error re…
Jul 15, 2015
cf110bc
fixed reference error and pep8 bugs
Jul 15, 2015
c9e5c5f
commented out tests and fixed test result output
Jul 16, 2015
5272381
printing of the pr results
Jul 16, 2015
528279a
fixed strip issue with PR results, from strip => rstrip
Jul 16, 2015
6a9ed32
run-tests-jenkins now properly runs the tests, removed license from f…
Jul 16, 2015
22f9be8
reverted license
Jul 16, 2015
5d63bc5
removed mima to fail binary compatibility checks
Jul 16, 2015
33e3143
reverted mima excludes
Jul 16, 2015
d82567a
set accumulator test suite to fail
Jul 16, 2015
2117c64
reverted accumulator test in favor of graphx test to only trigger 'gr…
Jul 16, 2015
831376e
fixed scala style check
Jul 16, 2015
d76f559
fixed scala style checks
Jul 16, 2015
05d9ef4
reverted scala test failure, added pyspark failure
Jul 16, 2015
0ad51b4
updated to add --absolute-names with tar command for archiving logs
Jul 27, 2015
a218baf
reverting python test
Aug 4, 2015
2853b60
fixed merge conflicts
Oct 16, 2015
fe397fb
updated to account for PR #7883
Oct 16, 2015
206930e
removed 'send_archived_logs' function
Oct 16, 2015
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
run-tests-jenkins now properly runs the tests, removed license from f… 
…ile to check RAT tests
  • Loading branch information
Brennon York committed Jul 16, 2015
commit 6a9ed32b487f30b1b7762324fb016d3af7a79b1e
16 changes: 0 additions & 16 deletions core/src/main/scala/org/apache/spark/Accumulators.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,19 +1,3 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package org.apache.spark

Expand Down
10 changes: 3 additions & 7 deletions dev/run-tests-jenkins.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -153,11 +153,9 @@ def run_tests(tests_timeout):
@return a tuple containing the test result code and the result note to post to Github
"""

# TODO: turn tests back on
# test_result_code = subprocess.Popen(['timeout',
# tests_timeout,
# os.path.join(SPARK_HOME, 'dev', 'run-tests')]).wait()
test_result_code = 0
test_result_code = subprocess.Popen(['timeout',
tests_timeout,
os.path.join(SPARK_HOME, 'dev', 'run-tests')]).wait()

failure_note_by_errcode = {
1: 'executing the `dev/run-tests` script', # error to denote run-tests script failures
Expand Down Expand Up @@ -245,8 +243,6 @@ def main():

test_result_code, test_result_note = run_tests(tests_timeout)

print("PR results:", pr_check_results)

# post end message
result_message = github_message('has finished')
Copy link
Contributor

@JoshRosen JoshRosen Jul 14, 2015

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here, it would be nice to have a template that uses string formatting / interpolation.

result_message += '\n' + test_result_note + '\n'
Expand Down