Skip to content

[SPARK-32647][INFRA] Report SparkR test results with JUnit reporter #29456

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
HyukjinKwon wants to merge 5 commits into from
Closed

[SPARK-32647][INFRA] Report SparkR test results with JUnit reporter #29456

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
c040c77
Use testthat's JUnit reporter
HyukjinKwon Aug 17, 2020
8fb651a
Add test failures
HyukjinKwon Aug 18, 2020
5bca181
Revert "Add test failures"
HyukjinKwon Aug 18, 2020
603268e
install xml2 in AppVeyor
HyukjinKwon Aug 18, 2020
1fe2806
Empty committer to kick AppVeyor again
HyukjinKwon Aug 18, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 6 additions & 3 deletions R/pkg/tests/run-all.R
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -61,15 +61,18 @@ if (identical(Sys.getenv("NOT_CRAN"), "true")) {
set.seed(42)

# TODO (SPARK-30663) To be removed once testthat 1.x is removed from all builds
if (grepl("^1\\..*", packageVersion("testthat"))) {
if (packageVersion("testthat")$major <= 1) {
# testthat 1.x
test_runner <- testthat:::run_tests
reporter <- "summary"

} else {
# testthat >= 2.0.0
test_runner <- testthat:::test_package_dir
reporter <- testthat::default_reporter()
dir.create("target/test-reports", showWarnings = FALSE)
reporter <- MultiReporter$new(list(
SummaryReporter$new(),
JunitReporter$new(file = "target/test-reports/test-results.xml")
Copy link
Member

@viirya viirya Aug 18, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it only supported by testthat 2.0+?

Copy link
Member Author

@HyukjinKwon HyukjinKwon Aug 18, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks supported in testthat 1.x but testthat 1.x support here is supposed to be removed soon at SPARK-30663. I didn't bother test as Jenkins and GitHub Actions use testthat 2+ anyway.

Comment on lines +73 to +74
Copy link
Member

@viirya viirya Aug 18, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need both SummaryReporter and JunitReporter?

Copy link
Member Author

@HyukjinKwon HyukjinKwon Aug 18, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yup, otherwise, the console output is not shown. I manually tested.

It was previously ProgressReporter (from testthat 2+) by default. It shows the console output such as:

��� |  0       | SparkSQL functions
��� |  1       | SparkSQL functions
��� |  2       | SparkSQL functions
...

in Jenkins (ever since we switched the default system encoding if I am not mistaken).

Now it follows the default format SummaryReporter (in testthat 1.x). It shows the console output as below:

functions in client.R: .....
test functions in sparkR.R: ..............................................
include R packages: ..
JVM API: ..
MLlib classification algorithms, except for tree-based algorithms: ...........................................................................

There are no differences in the report format when some tests are failed.

))
}

test_runner("SparkR",
Expand Down
4 changes: 2 additions & 2 deletions appveyor.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,8 +41,8 @@ cache:
install:
# Install maven and dependencies
- ps: .\dev\appveyor-install-dependencies.ps1
# Required package for R unit tests
- cmd: Rscript -e "install.packages(c('knitr', 'rmarkdown', 'testthat', 'e1071', 'survival', 'arrow'), repos='https://cloud.r-project.org/')"
# Required package for R unit tests. xml2 is required to use jUnit reporter in testthat.
- cmd: Rscript -e "install.packages(c('knitr', 'rmarkdown', 'testthat', 'e1071', 'survival', 'arrow', 'xml2'), repos='https://cloud.r-project.org/')"
- cmd: Rscript -e "pkg_list <- as.data.frame(installed.packages()[,c(1, 3:4)]); pkg_list[is.na(pkg_list$Priority), 1:2, drop = FALSE]"

build_script:
Expand Down