Skip to content

Conversation

@individual-it
Copy link
Member

Description

the exit code of behat running selenium tests was not checked correctly
this resulted in showing failed test as passed in travis & saucelabs

Related Issue

#27879

How Has This Been Tested?

locally & by cherry picking the commits to #27872

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

  • My code follows the code style of this project.
  • My change requires a change to the documentation.
  • I have updated the documentation accordingly.
  • I have read the CONTRIBUTING document.
  • I have added tests to cover my changes.
  • All new and existing tests passed.

@phil-davis
Copy link
Contributor

phil-davis commented May 15, 2017

Here is a test build without this change: https://travis-ci.org/individual-it/owncloud-core/builds/232406044?utm_source=github_status&utm_medium=notification
I purposely made a test that would fail, and this Job shows the scenario failure in it: https://travis-ci.org/individual-it/owncloud-core/jobs/232406052
but the status is not passed back, and Travis thinks that the Job and thus the whole Build passed, when actually it failed.

Here is a test build with the same bad test, and this change to pass back the status properly: https://travis-ci.org/individual-it/owncloud-core/jobs/232406052
Each of the 4 selenium jobs show as failed (as they should), e.g.: https://travis-ci.org/individual-it/owncloud-core/jobs/232408321

So this is "a good thing".

@phil-davis
Copy link
Contributor

Hmm @individual-it why did Job https://travis-ci.org/owncloud/core/jobs/232354259 fail with messages about scenarios "Failed asserting that two strings are equal."?

@individual-it
Copy link
Member Author

restarting the job did fix it

@phil-davis
Copy link
Contributor

Always good to know that the traditional support "fix" even works here - restart or reboot ;)

Copy link
Contributor

@PVince81 PVince81 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@PVince81 PVince81 merged commit 81d2f35 into master May 15, 2017
@PVince81 PVince81 deleted the fix-check-behat-exit branch May 15, 2017 14:03
@lock
Copy link

lock bot commented Aug 3, 2019

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

@lock lock bot locked as resolved and limited conversation to collaborators Aug 3, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants