Skip to content

js-red-check-accepts-a-thrown-error/1: The Node red check accepts a failing test with a plain Error - #985

Merged
bwmatson merged 5 commits into
mainfrom
spec/js-red-check-accepts-a-thrown-error/1
Nov 17, 2026
Merged

bwmatson merged 5 commits into
mainfrom
spec/js-red-check-accepts-a-thrown-error/1

Conversation

@bwmatson

Copy link
Copy Markdown
Owner

Start here

Already verified by the pipeline

  • Tests were committed first and seen to fail first.
  • unit-test result: passed
  • Review took 2 review rounds

Review tier: light — the reviewer classified it light

Why

The Node profile judges a tests commit red only when a failing test's JUnit type is AssertionError, or its message is exactly not implemented. A test that calls code which throws a plain Error of its own is rejected as "not one we recognise as 'the behaviour isn't there yet'".

loader-knows-the-usage-dataset/1 hit it: its three new loader tests failed correctly, with Dataset "usage" is not a report dataset thrown by the loader, and the red check rejected them three times. The unit only pa

… The full reason is in openspec/changes/js-red-check-accepts-a-thrown-error/proposal.md.

What this pull request does

The Node toolchain's red check now accepts a test that fails because the code under test throws a plain Error. It still refuses TypeError, ReferenceError and SyntaxError failures and names the type in the reason.

  • interpret_red in the Node profile treats a failure of type Error as red, the same as an AssertionError.
  • A failure of type TypeError, ReferenceError or SyntaxError is rejected with a reason that names the type and says it points to a missing name or a broken test, not missing behaviour.
  • Recorded Vitest JUnit reports (plain Error, each refused error type, and an error beside a passing test) and the script that records them back the new red-check tests.
  • The toolchain-profiles doc and a changelog bullet describe the new rule.
  • The replay test for a build without the driver's environment now checks the SKIPPED lines and the child run's exit status instead of searching the whole output for 'error'. A worktree path that contains that word no longer makes it fail.

Based on main, with nothing unmerged beneath it — ready to merge whenever the checks look right.

Unit js-red-check-accepts-a-thrown-error/1 of change js-red-check-accepts-a-thrown-error, task group(s) 1.
Spec: openspec/changes/js-red-check-accepts-a-thrown-error/ in the planning repo.

Assumptions

This unit assumes nothing unmerged: everything it builds on is already in main.

Verification

This is a unit-test unit: everything it needs is fakes, fixtures or throwaway containers, so GitHub Actions runs the whole of it. No e2e-test run was required.

How this was built

Tests were committed first and seen to fail before any implementation existed; the commit order is checked mechanically; lint, formatting and types pass at the tip.


Opened by the spec-driven pipeline. The build agent never merges. The pipeline merges only
a unit review classified auto and policy allowed; a person merges any other, after checking
the stack order above.

The tests come first and fail until the implementation.

The Node profile judges a tests commit red only when a failing test's JUnit `type` is `AssertionError`, or its message is exactly `not implemented`.

Unit: js-red-check-accepts-a-thrown-error/1
With this in, the tests of the previous commit pass.

The Node profile judges a tests commit red only when a failing test's JUnit `type` is `AssertionError`, or its message is exactly `not implemented`.

Unit: js-red-check-accepts-a-thrown-error/1
Unit: js-red-check-accepts-a-thrown-error/1
…view round 1)

Unit: js-red-check-accepts-a-thrown-error/1
@bwmatson bwmatson added change-js-red-check-accepts-a-thrown-error The change this pull request belongs to checking In review, but its checks are still running review-light Review asks for a light look from a person in-review Built and waiting for human review and removed checking In review, but its checks are still running labels Nov 17, 2026
@bwmatson
bwmatson merged commit e7c8311 into main Nov 17, 2026
5 checks passed
@bwmatson bwmatson added settling Merged; waiting for its checks to complete and removed in-review Built and waiting for human review settling Merged; waiting for its checks to complete labels Nov 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

change-js-red-check-accepts-a-thrown-error The change this pull request belongs to review-light Review asks for a light look from a person

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant