⚠️ No Traditional Unit Tests: ExcelMcp has no unit tests. Integration tests ARE our unit tests because Excel COM cannot be meaningfully mocked. Seedocs/ADR-001-NO-UNIT-TESTS.mdfor full architectural rationale.
# Development (fast feedback - excludes VBA tests)
dotnet test --filter "Category=Integration&RunType!=OnDemand&Feature!=VBA&Feature!=VBATrust"
# Pre-commit (comprehensive - excludes VBA tests)
dotnet test --filter "Category=Integration&RunType!=OnDemand&Feature!=VBA&Feature!=VBATrust"
# Session/batch changes (MANDATORY when modifying session/batch code)
dotnet test --filter "RunType=OnDemand"
# VBA tests (manual only - requires VBA trust enabled)
dotnet test --filter "(Feature=VBA|Feature=VBATrust)&RunType!=OnDemand"For complete testing guidance, see:
- Testing Strategy - Quick reference, templates, common mistakes
- Critical Rules - Mandatory development rules (Rule 14: SaveAsync)
tests/
├── ExcelMcp.Core.Tests/ # Core business logic (Integration)
├── ExcelMcp.Diagnostics.Tests/ # Excel COM behavior research (OnDemand, Manual)
├── ExcelMcp.McpServer.Tests/ # MCP protocol layer (Integration)
├── ExcelMcp.CLI.Tests/ # CLI wrapper (Integration)
└── ExcelMcp.ComInterop.Tests/ # COM utilities (OnDemand)
llm-tests/ # LLM tool behavior validation (Manual)
| Category | Speed | Requirements | Run By Default |
|---|---|---|---|
| Integration | Medium (10-20 min) | Excel + Windows | ✅ Yes (local) |
| OnDemand | Slow (3-5 min) | Excel + Windows | ❌ No (explicit only) |
| Diagnostics | Slow (varies) | Excel + Windows | ❌ No (manual, excluded from CI) |
| LLM Tests | Slow (varies) | Excel + Azure OpenAI | ❌ No (manual only) |
Diagnostics tests are research/exploratory tests in ExcelMcp.Diagnostics.Tests that document the actual behavior of Excel's COM APIs without our abstraction layer. These tests are excluded from CI to keep automation focused on core functionality.
Purpose:
- Understand Excel COM API behavior for Power Query, Data Model, PivotTables, etc.
- Document findings and edge cases for future implementation decisions
- Test alternative approaches to complex Excel operations
Trait markers:
Layer=DiagnosticsRunType=OnDemand
Run diagnostics tests locally:
# All diagnostics tests
dotnet test tests/ExcelMcp.Diagnostics.Tests/ --filter "RunType=OnDemand&Layer=Diagnostics"
# Specific diagnostic tests
dotnet test tests/ExcelMcp.Diagnostics.Tests/ --filter "Feature=PowerQuery&RunType=OnDemand"CI Behavior:
- Diagnostics tests are NOT run in CI workflows (GitHub Actions)
- Path filter includes folder to trigger builds when tests change
- Test execution uses
RunType!=OnDemandfilter to exclude them
# Test specific feature only
dotnet test --filter "Feature=PowerQuery&RunType!=OnDemand"
dotnet test --filter "Feature=DataModel&RunType!=OnDemand"
dotnet test --filter "Feature=Tables&RunType!=OnDemand"
dotnet test --filter "Feature=PivotTables&RunType!=OnDemand"
dotnet test --filter "Feature=Ranges&RunType!=OnDemand"
dotnet test --filter "Feature=Connections&RunType!=OnDemand"| Scenario | Command |
|---|---|
| Daily development | dotnet test --filter "Category=Integration&RunType!=OnDemand&Feature!=VBA" |
| Before commit | dotnet test --filter "Category=Integration&RunType!=OnDemand&Feature!=VBA" |
| Modified session/batch code | dotnet test --filter "RunType=OnDemand" (see Rule 3) |
| VBA development | dotnet test --filter "(Feature=VBA|Feature=VBATrust)&RunType!=OnDemand" |
| LLM behavior validation | See LLM Tests section below |
The llm-tests/ project validates that LLMs correctly use Excel MCP Server and CLI tools using pytest-skill-engineering.
- Manual/on-demand only - Not part of CI/CD
- After changing tool descriptions or adding new tools
- To validate LLM behavior patterns (e.g., incremental updates vs rebuild)
# From llm-tests/
uv sync
uv run pytest -m aitest -vAZURE_OPENAI_ENDPOINTenvironment variable- Windows desktop with Excel installed
- GitHub auth via
gh auth loginorGITHUB_TOKEN
See LLM Tests README for complete documentation.
VBA tests are excluded from normal test runs because:
- Stable codebase - VBA features are mature with minimal changes
- Performance - Excluding VBA tests makes integration tests ~25% faster (10-15 min vs 15-20 min)
- Special requirements - VBA tests require VBA trust enabled in Excel settings
- Opt-in model - Explicit testing when VBA code changes, rather than every commit
Run VBA tests manually when:
- Modifying VBA-related code (ScriptCommands, VbaTrustDetection)
- Adding new VBA features
- Before releasing VBA-related changes
- Troubleshooting VBA-specific issues
# Run ONLY VBA tests
dotnet test --filter "(Feature=VBA|Feature=VBATrust)&RunType!=OnDemand"
# Run ALL tests including VBA (takes longer)
dotnet test --filter "Category=Integration&RunType!=OnDemand"All VBA tests are tagged with [Trait("Feature", "VBA")] or [Trait("Feature", "VBATrust")]:
tests/ExcelMcp.Core.Tests/Integration/Commands/Script/
- ScriptCommandsTests.cs
- ScriptCommandsTests.Lifecycle.cs
- VbaTrustDetectionTests.ScriptCommands.cs
- VbaTrustDetectionTests.cs
tests/ExcelMcp.CLI.Tests/Integration/Commands/
- ScriptAndSetupCommandsTests.cs
VBA tests require VBA trust enabled in Excel:
# Enable VBA trust (required for VBA tests)
Set-ItemProperty -Path "HKCU:\Software\Microsoft\Office\16.0\Excel\Security" -Name "AccessVBOM" -Value 1
# Verify setting
Get-ItemProperty -Path "HKCU:\Software\Microsoft\Office\16.0\Excel\Security" -Name "AccessVBOM"Security Note: Only enable VBA trust in development environments. Production systems should keep this disabled.
- ✅ File Isolation - Each test creates unique file (no sharing)
- ✅ Binary Assertions - Pass OR fail, never "accept both"
- ✅ Verify Excel State - Always verify actual Excel state after operations
- ❌ No SaveAsync - Unless testing persistence (see Rule 14)
- Test failures: Check test output for detailed error messages
- Excel issues: Ensure Excel 2016+ installed and activated
- Session/batch issues: Run OnDemand tests to verify cleanup
- Writing tests: See Testing Strategy