Skip to content

CI: one test binary per crate, nextest, line tables only - #1213

Merged
aiongg merged 3 commits into
storytold:mainfrom
aiongg:ci-speed
Oct 23, 2026
Merged

aiongg merged 3 commits into
storytold:mainfrom
aiongg:ci-speed

Conversation

@aiongg

@aiongg aiongg commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

Makes cargo xtask ci faster and its target dir smaller, following the measurements in storytold/craft-common-actions#60.

Measured

One run of cargo xtask ci on a 16-core Linux box, starting from an empty target dir (all dependencies built from scratch), through a two-slot build queue (8 jobs per build). All 8 steps passed.

Sum of the cargo steps about 10 min: clippy 2:03, test build 6:16, wasm check 1:15, the rest under a minute
Wall clock of the command 30:30, of which about 20 min was waiting for a free build slot on a shared box
Target dir after the run 7.5 GB with incremental on (deps 4.0 GB, incremental 3.1 GB); 4.0 GB with CARGO_INCREMENTAL=0, as in CI
Test binaries (nextest) 33
Integration-test binaries 5, from 35 files
Test run 1741 tests (6 skipped) in 11.8 s with nextest

There is no earlier run for comparison: this repository was not part of the study, and no "before" build was made.

Changes

  • Profiles. [profile.dev] debug = "line-tables-only" for workspace crates and debug = false for dependencies. Panics keep their file and line. The opt-levels stay as they were.
  • One integration-test binary per crate. Each crate's tests/*.rs files become modules of tests/it/main.rs: crates/pptx (27 files), apps/deckcraft-cli (4), crates/engine (2), crates/mcp (1), crates/media (1).
    • The #![allow(clippy::unwrap_used, ...)] inner attributes moved to the mod lines of automation_roots and mcp_roots.
    • tests/fixtures stays in place.
    • Docs that name a test file (docs/*.md) point at the new paths.
  • xtask ci.
    • Tests run with cargo nextest run --workspace --lib --bins --tests when nextest is installed, and with cargo test otherwise (with a one-line notice). Doctests run separately with cargo test --workspace --doc. cargo xtask test runs both.
    • The test build leaves out examples; clippy (--all-targets) still checks them.
    • The wasm check is one cargo check --target wasm32-unknown-unknown --keep-going call over the same 23 crates, instead of one call per crate.
    • Clippy runs with --keep-going. Clippy and the test build share only dependency build scripts and proc macros: clippy builds metadata, tests build machine code.

ci.yml is not changed here; a shared workflow replaces it separately. No rust-toolchain.toml: the shared release workflows install their cross targets on stable, so a pinned channel would leave those jobs without their targets.

Slowest tests

All 1741 tests run in 11.8 s. The slowest, measured while the box was loaded: pptx malformed::fuzz_mutations 9.5 s (a fixed-seed mutation sweep, already 300 iterations in debug builds), engine soak_tests::soak_random_edits_keep_the_document_valid 4.2 s, ui-egui ui_soak_tests::ui_soak_random_input_keeps_the_app_sound 4.2 s, render transition::tests::every_transition_starts_old_and_ends_new 3.8 s, pptx malformed::bad_xml_in_each_part 2.9 s. They are fuzz and soak sweeps; none runs more inputs than its assertion needs, so they are unchanged.

Summary by CodeRabbit

  • Developer Tools
    • Added separate commands for running tests and documentation tests. Test runs use nextest when available and fall back to Cargo’s standard test runner.
    • WASM checks now validate selected packages together, and CI runs Clippy before tests and documentation tests.
    • Updated development build settings to reduce debug information in dependencies while retaining line information for workspace code.
  • Documentation
    • Updated test and file-format references to reflect the current integration-test organization.

@coderabbitai

coderabbitai Bot commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
📥 Commits

Reviewing files that changed from the base of the PR and between b47195f and 1bf1ae5.

📒 Files selected for processing: 47
  • AGENTS.md
  • Cargo.toml
  • apps/deckcraft-cli/tests/it/cli.rs
  • apps/deckcraft-cli/tests/it/convert.rs
  • apps/deckcraft-cli/tests/it/damaged.rs
  • apps/deckcraft-cli/tests/it/main.rs
  • apps/deckcraft-cli/tests/it/mcp_roots.rs
  • crates/engine/tests/it/chart_native.rs
  • crates/engine/tests/it/main.rs
  • crates/engine/tests/it/show_clicks.rs
  • crates/mcp/tests/it/automation_roots.rs
  • crates/mcp/tests/it/main.rs
  • crates/media/tests/it/decode.rs
  • crates/media/tests/it/main.rs
  • crates/pptx/tests/it/anchor.rs
  • crates/pptx/tests/it/chart_embed.rs
  • crates/pptx/tests/it/comments.rs
  • crates/pptx/tests/it/corpus.rs
  • crates/pptx/tests/it/corpus_charts.rs
  • crates/pptx/tests/it/corpus_elements.rs
  • crates/pptx/tests/it/corpus_kinds.rs
  • crates/pptx/tests/it/corpus_media.rs
  • crates/pptx/tests/it/corpus_style.rs
  • crates/pptx/tests/it/corpus_themes.rs
  • crates/pptx/tests/it/fills.rs
  • crates/pptx/tests/it/kept_parts.rs
  • crates/pptx/tests/it/locks.rs
  • crates/pptx/tests/it/main.rs
  • crates/pptx/tests/it/malformed.rs
  • crates/pptx/tests/it/math_ns.rs
  • crates/pptx/tests/it/notes.rs
  • crates/pptx/tests/it/ole.rs
  • crates/pptx/tests/it/ole_vml.rs
  • crates/pptx/tests/it/placeholders.rs
  • crates/pptx/tests/it/roundtrip.rs
  • crates/pptx/tests/it/sample.rs
  • crates/pptx/tests/it/smartart.rs
  • crates/pptx/tests/it/sniff.rs
  • crates/pptx/tests/it/table_styles.rs
  • crates/pptx/tests/it/tags.rs
  • crates/pptx/tests/it/transition_xml.rs
  • crates/pptx/tests/it/trigger_steps.rs
  • docs/file-format-parity.md
  • docs/gaps.md
  • docs/parity-checklist.md
  • docs/roadmap.md
  • xtask/src/main.rs
💤 Files with no reviewable changes: 2
  • apps/deckcraft-cli/tests/it/mcp_roots.rs
  • crates/mcp/tests/it/automation_roots.rs

Limit details: You’ve used all 10 included reviews currently available.

📝 Walkthrough

The changes update development debug profiles, declare integration-test modules, and revise test-path references. The xtask commands now support nextest selection, separate doctests, consolidated wasm checks, and an updated CI sequence.

Changes: Test and CI workflow

Layer / File(s) Summary
Development debug profiles
Cargo.toml
The wildcard development-package profile disables debug information. The workspace development profile retains line tables.
Integration-test entry points and references
apps/deckcraft-cli/tests/it/*, crates/engine/tests/it/main.rs, crates/mcp/tests/it/*, crates/media/tests/it/main.rs, crates/pptx/tests/it/main.rs, docs/file-format-parity.md, docs/gaps.md, docs/parity-checklist.md, docs/roadmap.md
Integration-test entry points declare additional modules and centralize selected Clippy allowances. Documentation references the integration-test locations.
xtask test and CI commands
xtask/src/main.rs, AGENTS.md
The test command uses nextest when available and otherwise falls back to cargo test. CI runs clippy, tests, and doctests; wasm checks use one workspace-level cargo check.

Priority: ⬇️ Low

Estimated code review effort: 3 (Moderate) | ~25 minutes

Change: Other

Sequence Diagram(s)

sequenceDiagram
  participant CargoXtaskCI
  participant Clippy
  participant TestCommand
  participant Nextest
  participant CargoTest
  participant Doctests
  CargoXtaskCI->>Clippy: run with --keep-going
  CargoXtaskCI->>TestCommand: run workspace tests
  TestCommand->>Nextest: check availability
  alt nextest is available
    TestCommand->>Nextest: run selected test targets
  else nextest is unavailable
    TestCommand->>CargoTest: run selected test targets
  end
  CargoXtaskCI->>Doctests: run workspace doctests
Loading

Possibly related issues

Suggested reviewers: echelon, isaac-levine

Merge Risk: ⚪ Minimal · up to 1bf1a

The updated CI and test workflow is ready to merge with normal checks.

Pre-merge checks: Passed 4 · Failed 1

Failed Failed checks: 1 warning

Check name Status Explanation Resolution
Docstring Coverage Warning Docstring coverage is 42.86% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 7 functions across 6 files. (6 skipped: 6… Write docstrings for the functions missing them to satisfy the coverage threshold.
Passed Passed checks: 4 passed
Check name Status Explanation
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check Passed The title "CI: one test binary per crate, nextest, line tables only" names three of the main changes in the PR: one integration-test binary per crate, nextest for CI test execution, and line-table deb…
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Full details: Title check

Explanation

The title "CI: one test binary per crate, nextest, line tables only" names three of the main changes in the PR: one integration-test binary per crate, nextest for CI test execution, and line-table debug info. The objectives confirm these are central, along with the wasm consolidation and clippy changes that the title does not mention. The title is specific and short, and it matches the changeset.

Full details: Docstring Coverage

Explanation

Docstring coverage is 42.86% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 7 functions across 6 files. (6 skipped: 6 unsupported.)

Resolution

Write docstrings for the functions missing them to satisfy the coverage threshold.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Usage-based review receipt

  • Mode: Continue automatically
  • Reviewed files: 12
  • Waived: $3.00 (charged $0.00)
  • View usage details

Note

This review exceeded your plan’s limits and used usage-based reviews—free during trial, billed after paid activation unless disabled. Manage usage-based reviews.

Comment @coderabbitai help to get the list of available commands.

@skylartaylor skylartaylor left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me, thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants