Skip to content
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Next Next commit
added docs for new visual tests
  • Loading branch information
Vaivaswat2244 committed Mar 17, 2025
commit 1f6eabed66fe9ec1dfb1b99cf3da8b422a4f1e62
Binary file added contributor_docs/images/pixelmatch.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added contributor_docs/images/pixelmatch2.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
41 changes: 41 additions & 0 deletions contributor_docs/unit_testing.md
Original file line number Diff line number Diff line change
Expand Up @@ -249,3 +249,44 @@ visualSuite('3D Model rendering', function() {
});
```


Different operating systems and browsers render graphics with subtle variations. These differences are normal and shouldn't cause tests to fail.
Common acceptable differences include:

* Single-pixel shifts in line positions
* Slight variations in anti-aliasing
* Text rendering differences (especially font weight and kerning)
* Minor differences in curve smoothness

For example, text rendered on macOS might appear slightly different from text rendered in a Linux CI environment. The same applies to thin lines, curves, and other graphical elements with anti-aliasing.
An example of this can be the below image which earlier caused tests to fail in CI because of different rendering environments.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We might want an additional sentence to reiterate that although this failed before, this is an example of something that should pass, and will pass using the method you describe below.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure

Copy link
Author

@Vaivaswat2244 Vaivaswat2244 Mar 20, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@davepagurek, Have made the changes

![Example](./images/pixelmatch2.png)

The p5.js visual testing system uses a sophisticated algorithm to distinguish between acceptable rendering variations and actual bugs:

* Initial comparison - Compares pixels with a moderate threshold (0.5) to identify differences using [pixelmatch](https://github.com/mapbox/pixelmatch) library for pixel to pixel comparison.
* Cluster identification - Groups connected difference pixels using a Breadth-First Search (BFS) algorithm
* Pattern recognition - The algorithm specifically identifies:

- "Line shift" clusters - differences that likely represent the same visual element shifted by 1px
- Isolated pixel differences (noise)


* Smart failure criteria - Applies different thresholds:

- Ignores clusters smaller than 4 pixels
- Allows up to 40 total significant difference pixels
- Permits minor line shifts that are typical across platforms

The below is the example of the tests that should fail:
![Example](./images/pixelmatch.png)



This approach balances sensitivity to real bugs while tolerating platform-specific rendering variations. The algorithm uses these key parameters:
```js
const MIN_CLUSTER_SIZE = 4; // Minimum significant cluster size
const MAX_TOTAL_DIFF_PIXELS = 40; // Maximum allowed significant differences
```
The algorithm identifies line shifts by analyzing the neighborhood of each difference pixel. If more than 80% of pixels in a cluster have ≤2 neighbors, it's classified as a line shift rather than a structural difference.
This intelligent comparison ensures tests don't fail due to minor rendering differences while still catching actual visual bugs.