[🚀 Feature]: Java Test ExpectedFailure Annotation

[titusfortner]

Description

Java tests currently use a mix of mechanisms for known browser-specific failures:

• @ignore skips a test for a browser.
• @NotYetImplemented acts like an xfail: matching failures are allowed, but the test fails once it unexpectedly passes.
• assumeThat(...) is commonly used inside tests or setup code to guard browser versions, platforms, or capability availability.

The assumeThat(...) pattern works for skipping unsupported combinations, for example Firefox version guards in DevTools tests, but it does not give the same cleanup signal as
@NotYetImplemented. If the guarded browser/version/platform starts working, the test remains skipped instead of failing and prompting removal of the guard.

Proposal: add a general repeatable @expectedfailure annotation that supports browsers, browser versions, platforms, reason, and issue reference:

@expectedfailure(
browsers = {CHROME},
platforms = {Platform.MAC},
minVersion = "149",
maxVersion = "149",
reason = "BiDi network response event is missing request body",
issue = "https://issues.chromium.org/..."
)

This would keep xfail semantics:

• If the annotation matches and the test fails, treat it as expected.
• If the annotation matches and the test passes, fail the test so the annotation gets removed.
• If the annotation does not match, failures behave normally.

For browser versions, pinned browser Bazel test targets already know what they are going to run. Java test setup could expose that value as a JVM property, such as
selenium.browser.version, so the JUnit extension can evaluate version-scoped expected failures before or during test execution.

Have you considered any alternatives or workarounds?

Current alternatives all have tradeoffs:

• assumeThat(...) can guard browser versions, platforms, and capabilities, but it skips the test and does not alert us when the condition is fixed.
• @ignore supports reason and issue, but also skips the test entirely.
• @NotYetImplemented gives the desired xfail/pass-now-fails behavior, but only supports browser matching and cannot express browser version, platform, or issue.
• Manual try/catch expected-failure logic inside tests can work, but it is noisy and inconsistent compared to centralizing this behavior in the Selenium test extension.

Extending @NotYetImplemented is possible, but a more general @expectedfailure name better describes browser bugs, beta regressions, platform-specific failures, and not-yet-
implemented behavior.