Update screenshot collector

[tkattkat]

why

• when transitioning to v3, we did not use the latest version of screenshot collector
• screenshot collector currently fails due to not having page.on and page.off support for the load, and domcontentloaded events.

what changed

• added latest version of screenshot collector

test plan

• ran evals in cli with additional logging to also verify everything is working as expected

---

Summary by cubic

Updated the evals CLI screenshot collector to the latest version, adding image-diff filtering and compatibility with v3 Stagehand pages where navigation events are disabled by default. This reduces duplicate screenshots and stabilizes capture during eval runs.

• New Features

  • Skip similar screenshots using MSE/SSIM thresholds with sharp.
  • Non-blocking initial/final captures and safer interval capture with error handling.
  • Added addScreenshot method for manual collection.

• Dependencies

  • Added sharp ^0.34.5 for image processing.
  • Patch bump via changeset for @browserbasehq/stagehand-evals.

^{Written for commit cb952c4. Summary will update automatically on new commits.}

[changeset-bot]

🦋 Changeset detected

Latest commit: cb952c4

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package

Name	Type
@browserbasehq/stagehand-evals	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[cubic-dev-ai]

No issues found across 4 files

[greptile-apps]

Greptile Overview

Greptile Summary

Updates the ScreenshotCollector utility for V3 compatibility and adds intelligent screenshot deduplication.

• Switched from generic ScreenshotCapablePage interface to the concrete V3 Page type from @browserbasehq/stagehand
• Added sharp library for image processing to enable screenshot deduplication via MSE (Mean Squared Error) and SSIM (Structural Similarity Index) comparisons
• Disabled navigation event listeners (page.on("load"), etc.) since V3 pages don't support these yet - documented with TODO comment
• Changed default captureOnNavigation from true to false to reflect current V3 limitations
• Added new addScreenshot() method for manual screenshot additions with deduplication logic
• Improved error handling with .catch() for async operations instead of fire-and-forget calls

Confidence Score: 4/5

• This PR is safe to merge with minor considerations around the async final screenshot behavior
• The changes are well-structured and address real V3 compatibility issues. The image deduplication logic is sound. One minor issue exists where the final screenshot in stop() may not be captured before returning, but this is unlikely to cause problems in practice since the collector is typically used for incremental captures.
• packages/evals/utils/ScreenshotCollector.ts - review async final screenshot handling in stop() method

Important Files Changed

File Analysis

Filename	Score	Overview
packages/evals/utils/ScreenshotCollector.ts	4/5	Major update adding image deduplication via MSE/SSIM, switching to V3 Page type, and adding sharp dependency for image comparison. Minor issue with async final screenshot not being awaited in stop().
packages/evals/package.json	5/5	Added sharp ^0.34.5 dependency for image processing used in screenshot deduplication.
.changeset/beige-taxes-punch.md	5/5	Standard changeset for patch update to stagehand-evals package.
pnpm-lock.yaml	5/5	Lock file updates for sharp 0.34.5 and its native dependencies across platforms.

Sequence Diagram

sequenceDiagram
    participant Caller
    participant ScreenshotCollector
    participant Page
    participant Sharp

    Caller->>ScreenshotCollector: start()
    ScreenshotCollector->>Page: screenshot() [initial]
    Page-->>ScreenshotCollector: Buffer
    ScreenshotCollector->>ScreenshotCollector: store screenshot
    
    loop Every interval (5000ms default)
        ScreenshotCollector->>Page: screenshot()
        Page-->>ScreenshotCollector: Buffer
        ScreenshotCollector->>Sharp: resize & compare (MSE)
        Sharp-->>ScreenshotCollector: MSE value
        alt MSE >= threshold
            ScreenshotCollector->>Sharp: calculate SSIM
            Sharp-->>ScreenshotCollector: SSIM value
            alt SSIM < threshold (different enough)
                ScreenshotCollector->>ScreenshotCollector: store screenshot
            end
        end
    end

    Caller->>ScreenshotCollector: stop()
    ScreenshotCollector->>Page: screenshot() [final, async]
    ScreenshotCollector-->>Caller: Buffer[]

Loading

[greptile-apps]

_{2 files reviewed, 1 comment}

_{Edit Code Review Agent Settings | Greptile}