Skip to content

test(signals): verify native decoder discovery coverage - #2818

Merged
morluto merged 17 commits into
mainfrom
review/signal-decoder-discovery-2487-20261013
Oct 13, 2026
Merged

morluto merged 17 commits into
mainfrom
review/signal-decoder-discovery-2487-20261013

Conversation

@N0zoM1z0

@N0zoM1z0 N0zoM1z0 commented Oct 13, 2026 •

Copy link
Copy Markdown
Collaborator

The real sigrok verifier previously checked only that UART appeared in a catalog search. CLI/MCP agreement could therefore pass when both adapters returned the same incomplete catalog or incorrect coverage counts.

Compare the installed native catalog and UART help independently with every returned page and all coverage fields. Exercise default and unlimited pages, offsets at/past the end, mixed-case queries, literal special characters, and empty matches. This follows up #2487; production behavior is unchanged.

Validation on Linux x64 with sigrok-cli 0.7.2 / libsigrokdecode 0.5.3, using 2 CPUs and an 8 GiB process-family limit:

  • 62/62 related tests pass on the synchronized base, covering provider/parser/cancellation/capacity/cleanup and converted CLI/MCP schemas; peak family RSS 3.16 GiB including a fresh compile.
  • Real native verifier: 21/21 cases on the synchronized base, including 11 discovery cases, CLI/MCP parity, exact catalog/help pages, coverage fields, and private-workspace cleanup; peak family RSS 1.27 GiB.
  • Four actual hardware recordings from sigrok-dumps at 0ad1347: STM32 UART greeting, 9-bit UART counter, FT232H/MAX7219 SPI, and Samsung EDID over I2C. All selected/full annotation pages match separate native commands; source SHA-256 values remain unchanged. This verifies existing recordings; no acquisition was performed.
  • Equivalent three-call UART discovery/help/greeting task, three alternating pairs on a warm MCP server: broad versus focused request/response counts are 14,382 vs 7,220 tokens; including the current complete input tool catalog, 198,283 vs 191,121 tokens (o200k_base, not billed usage; model reasoning excluded). Median wall time is 1.286 vs 1.367 seconds, CPU 0.92 vs 0.90 seconds, and sampled peak server/native process-family RSS 435 vs 434 MiB. Page cache is uncontrolled and the sampler adds overhead; no speedup or caching claim. The complete hardware/benchmark run peaked at 876 MiB.
  • npm run check:ci: 8/8 static tasks and architecture guards passed on the previously synchronized main; peak family RSS 4.13 GiB. On the final current-base head, it reaches the formatter and fails only in two unchanged main files (enhancedTools.04-uses-complete-inventories-inline.test.ts and packagePlatformDiagnostics.test.ts); both are byte-identical to origin/main. The run peaked at 2.60 GiB. git diff --check passes.

The validated revision is 6ab5a82d95f5eac85d2630d3c2035a395d921d4a. Windows/macOS sigrok behavior is not claimed; this provider is explicitly Linux-only.

Compare catalog and help pages against independent native sigrok output,
including full coverage counts, empty matches and boundary offsets.

Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 13, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-13T19:49:38.676322Z 6ab5a82 New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@greptile-apps

greptile-apps Bot commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerConfidence Score: 5/5

[Low impact] The PR appears safe to merge; no blocking issue was found.

Summary

Strengthens scripts/verify-signal-recording.mjs by comparing discovery results with a separately read native catalog and UART help.

  • The verifier checks decoder pages against native sigrok output.

Reviews (8) · Last reviewed commit: "chore: verify PR #2818 on current main" · Reviewed by Greptile

@morluto morluto added area: runtime Process capture and runtime observation area: build Build, CI, packaging, and publishing area: tests Test suite and verification P3: low Low priority, nice to have labels Oct 13, 2026
N0zoM1z0 and others added 3 commits October 13, 2026 14:18
Include the upstream formatting and test-function lint corrections before
rerunning decoder-discovery verification and protected merge checks.

Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
Include the upstream formatting and test-function lint corrections before
rerunning decoder-discovery verification and protected merge checks.

Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
dakjdakd and others added 13 commits October 14, 2026 00:57
* fix(comparison): preserve incomplete function call evidence

Keep missing, unknown, partial and unresolved call relationships unknown in function comparisons and reconstruction claims. Aggregate IDA caller and callee coverage without overwriting uncertainty.

Validate with 144 focused tests, typecheck/lint, docs checks and test discovery. Real providers were not verified; the full headless IDA session suite requires unavailable Windows native controls.

* fix(comparison): recheck historical calls-only changes

---------

Co-authored-by: morluto <76467478+morluto@users.noreply.github.com>
Preserve the maintainer formatting fix and current-main synchronization while retaining the native signal discovery verifier.

Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
Validate signal discovery coverage against the latest reconstruction fixture baseline.

Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
Unify the tested head with the maintainer branch after both incorporated the same current main revision.

Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
Revalidate the signal discovery verifier against the current integration baseline.

Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
Carry the verified signal discovery change over the latest unrelated application fixes.

Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
Validate the signal discovery verifier on the settled integration head.

Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
Include the latest unrelated package-verifier ownership fix before qualification.

Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
Include the latest unrelated Wasm and installation-documentation updates before the final CI run.

Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
Include the latest unrelated process-deadline and ZIP-coverage fixes before local qualification.

Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
@morluto
morluto merged commit 87a8c8c into main Oct 13, 2026
4 of 5 checks passed
@morluto
morluto deleted the review/signal-decoder-discovery-2487-20261013 branch October 13, 2026 19:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: build Build, CI, packaging, and publishing area: runtime Process capture and runtime observation area: tests Test suite and verification P3: low Low priority, nice to have

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants