Repository navigation
fix(reviewer): return success=False when QA finds issues - #166
kapil971390 wants to merge 1 commit into
Conversation
0xDevNinja
left a comment
There was a problem hiding this comment.
Thanks for this — the underlying concern is valid (a QA run that finds issues shouldn't look like a clean pass). But I think flipping success is the wrong lever, and it diverges from how the rest of the codebase treats QA results.
ToolResult.success means "the tool executed correctly", not "QA passed."
- The sibling QA tool follows this:
tools/analysis/visual_qa.pybuildsvalidation_issuesand still returnssuccess=True(around L277-281), reservingsuccess=Falsefor actual execution errors (L126, L138, L140). - The QA verdict is already surfaced in the report itself:
statusis set to"pass"/"revise"attools/character/character_animation.py:868, and the findings are inissues[](L883). - The pipeline already gates on that status, not on
success:skills/meta/reviewer.mdL291-306 makes its decisions offfinal_review.status(revise/pass), notToolResult.success.
Regression risk: several callers use if result.success: to mean "it ran, safe to read the data" (e.g. lib/source_media_review.py:50, 96, 201). With this change, a normal revise outcome returns success=False, so that pattern would treat a perfectly valid QA report as a failed run and skip it.
Suggested change: revert to success=True. If the problem is that the pipeline isn't stopping when QA returns revise, the fix belongs in the consumer/skill that should be honoring report["status"] — not in overloading success.
Happy to be corrected if there's a specific consumer that reads success as the QA verdict here — if so, can you point to it?
|
Thanks for the detailed review @0xDevNinja — you're right on all counts. I traced through the call chain more carefully:
Flipping Closing this PR. Thanks for taking the time to walk through the codebase. |
CharacterAnimationReviewer.execute()always returnedsuccess=Trueeven when the QA check found issues — so the compose-director had no way to know the rig was broken.Reproduction
Output before fix:
Fix
One-line change in
CharacterAnimationReviewer.execute():After fix:
The
successfield now correctly reflects whether QA passed, so downstream agents and compose-director can gate on it.