Skip to content

Conversation

@rmccorm4
Copy link
Contributor

@rmccorm4 rmccorm4 commented Nov 15, 2025

Overview:

  • Removes 100s of lines of spam in test output about unknown pytest markers
  • Runs tests in CI with the actual configurations we've defined in pyproject.toml as a single source of truth
    • This also led to treating warnings as errors, hence the addition of more "ignores" of certain warnings added in this PR.

Details:

  • The pytest sections of the Build&Test github actions have hundreds of lines of spam in the output about unknown pytest markers like this:
...
tests/kvbm_integration/test_determinism_disagg.py:39
  /workspace/tests/kvbm_integration/test_determinism_disagg.py:39: PytestUnknownMarkWarning: Unknown pytest.mark.slow - is this a typo?  You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/stable/how-to/mark.html
    pytest.mark.slow,

tests/kvbm_integration/test_determinism_disagg.py:40
  /workspace/tests/kvbm_integration/test_determinism_disagg.py:40: PytestUnknownMarkWarning: Unknown pytest.mark.gpu_2 - is this a typo?  You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/stable/how-to/mark.html
    pytest.mark.gpu_2,

tests/kvbm_integration/test_determinism_disagg.py:488
  /workspace/tests/kvbm_integration/test_determinism_disagg.py:488: PytestUnknownMarkWarning: Unknown pytest.mark.kvbm_v2 - is this a typo?  You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/stable/how-to/mark.html
    @pytest.mark.kvbm_v2
...
  • The solution is to register the custom marks in your configuration: https://docs.pytest.org/en/stable/how-to/mark.html#registering-marks
  • We already do register the custom marks in pyproject.toml here:

    dynamo/pyproject.toml

    Lines 183 to 207 in 1e120ed

    markers = [
    "pre_merge: marks tests to run before merging",
    "nightly: marks tests to run nightly",
    "weekly: marks tests to run weekly",
    "gpu_1: marks tests to run on GPU",
    "gpu_2: marks tests to run on 2GPUs",
    "gpu_4: marks tests to run on 4GPUs",
    "gpu_8: marks tests to run on 8GPUs",
    "e2e: marks tests as end-to-end tests",
    "integration: marks tests as integration tests",
    "unit: marks tests as unit tests",
    "stress: marks tests as stress tests",
    "vllm: marks tests as requiring vllm",
    "trtllm: marks tests as requiring trtllm",
    "trtllm_marker: marks tests as requiring trtllm",
    "sglang: marks tests as requiring sglang",
    "slow: marks tests as known to be slow",
    "h100: marks tests to run on H100",
    "kvbm: marks tests for KV behavior and model determinism",
    "kvbm_v2: marks tests using KVBM V2",
    "model: model id used by a test or parameter",
    "custom_build: marks tests that require custom builds or special setup (e.g., MoE models)",
    "k8s: marks tests as requiring Kubernetes",
    "fault_tolerance: marks tests as fault tolerance tests"
    ]
  • However, we don't copy the pyproject.toml into the container where pytest gets executed, so it doesn't see this configuration and thus outputs all the warnings about unknown markers.
  • This PR copies the pyproject.toml into the containers so the pytest configuration is respected and we don't see these warnings are not produced when running the pytests inside the container

Where should the reviewer start?

Related Issues: (use one of the action keywords Closes / Fixes / Resolves / Relates to)

  • closes GitHub issue: #xxx

Summary by CodeRabbit

  • Chores
    • Updated Docker container configurations for Sglang, TRTLLM, and VLLM to include project configuration files, enabling pytest utilities and test tooling within containers.

…f unknown pytest markers during pytest execution
@rmccorm4 rmccorm4 requested review from a team as code owners November 15, 2025 21:44
@github-actions github-actions bot added the chore label Nov 15, 2025
@rmccorm4 rmccorm4 requested a review from indrajit96 November 15, 2025 21:44
@coderabbitai
Copy link
Contributor

coderabbitai bot commented Nov 15, 2025

Walkthrough

Three container Dockerfiles are updated to include COPY commands that place pyproject.toml into the container workspace at /workspace/pyproject.toml. The file ownership is set to the dynamo user. This enables pytest configuration markers and related tooling to be accessible within the built container images.

Changes

Cohort / File(s) Change Summary
Container Dockerfile Updates
container/Dockerfile.sglang, container/Dockerfile.trtllm, container/Dockerfile.vllm
Added COPY steps to include pyproject.toml into workspace with --chown=dynamo flag in both runtime and development paths, enabling pytest configuration availability inside containers

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

  • Verify consistency of COPY syntax and paths across all three Dockerfiles
  • Confirm --chown=dynamo flag is correctly applied in each location
  • Validate that pyproject.toml file paths match across sglang, trtllm, and vllm images

Poem

🐰 Three containers need their config true,
Pyproject.toml now hops right through,
With dynamo's care, the workspace grows,
Pytest markers dance wherever it goes! 🎉

Pre-merge checks

✅ Passed checks (3 passed)
Check name Status Explanation
Title check ✅ Passed The title accurately summarizes the main change: copying pyproject.toml into containers to prevent pytest marker warnings.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Description check ✅ Passed The pull request description provides a clear overview, detailed explanation of the problem and solution, and identifies the specific files to review, but the 'Related Issues' section is incomplete.

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Nitpick comments (1)
container/Dockerfile.vllm (1)

290-295: Redundant copy and pattern inconsistency with sglang/trtllm.

The COPY . /workspace/ at line 292 already copies pyproject.toml, making the explicit copy at lines 294-295 redundant. Additionally, this broad-copy pattern differs from sglang and trtllm, which use selective COPY commands (tests, examples, benchmarks, etc.) before copying pyproject.toml. The FIXME comment acknowledges this architectural concern.

Consider either removing the redundant copy (since . includes it) or refactoring this stage to use selective copies for consistency with other framework Dockerfiles, per the learnings about maintaining uniformity across framework-specific setups.

📜 Review details

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 1e120ed and 3965a98.

📒 Files selected for processing (3)
  • container/Dockerfile.sglang (1 hunks)
  • container/Dockerfile.trtllm (1 hunks)
  • container/Dockerfile.vllm (1 hunks)
🧰 Additional context used
🧠 Learnings (5)
📓 Common learnings
Learnt from: keivenchang
Repo: ai-dynamo/dynamo PR: 2797
File: container/Dockerfile:437-449
Timestamp: 2025-08-30T20:43:49.632Z
Learning: In the dynamo project's devcontainer setup, the team prioritizes consistency across framework-specific Dockerfiles (like container/Dockerfile, container/Dockerfile.vllm, etc.) by mirroring their structure, even when individual optimizations might be possible, to maintain uniformity in the development environment setup.
Learnt from: keivenchang
Repo: ai-dynamo/dynamo PR: 2797
File: .devcontainer/devcontainer.json:12-12
Timestamp: 2025-08-30T20:43:10.091Z
Learning: In the dynamo project, devcontainer.json files use templated container names (like "dynamo-vllm-devcontainer") that are automatically processed by the copy_devcontainer.sh script to generate framework-specific configurations with unique names, preventing container name collisions.
Learnt from: keivenchang
Repo: ai-dynamo/dynamo PR: 2797
File: .devcontainer/devcontainer.json:12-12
Timestamp: 2025-08-30T20:43:10.091Z
Learning: In the dynamo project's devcontainer setup, hard-coded container names in devcontainer.json files serve as templates that are automatically processed by the copy_devcontainer.sh script to generate framework-specific configurations with unique names, preventing container name collisions.
Learnt from: keivenchang
Repo: ai-dynamo/dynamo PR: 3051
File: container/templates/Dockerfile.vllm.j2:221-233
Timestamp: 2025-09-16T17:16:07.820Z
Learning: In the dynamo project, when converting Dockerfiles to Jinja2 templates, the primary goal is backward compatibility - generated Dockerfiles must be identical to the originals. Security improvements or other enhancements that would change the generated output are out of scope for templating PRs and should be addressed separately.
Learnt from: keivenchang
Repo: ai-dynamo/dynamo PR: 2822
File: container/Dockerfile.vllm:343-352
Timestamp: 2025-09-03T01:10:12.599Z
Learning: In the dynamo project's local-dev Docker targets, USER_UID and USER_GID build args are intentionally left without default values to force explicit UID/GID mapping during build time, preventing file permission issues in local development environments where container users need to match host user permissions for mounted volumes.
📚 Learning: 2025-08-30T20:43:49.632Z
Learnt from: keivenchang
Repo: ai-dynamo/dynamo PR: 2797
File: container/Dockerfile:437-449
Timestamp: 2025-08-30T20:43:49.632Z
Learning: In the dynamo project's devcontainer setup, the team prioritizes consistency across framework-specific Dockerfiles (like container/Dockerfile, container/Dockerfile.vllm, etc.) by mirroring their structure, even when individual optimizations might be possible, to maintain uniformity in the development environment setup.

Applied to files:

  • container/Dockerfile.vllm
  • container/Dockerfile.trtllm
  • container/Dockerfile.sglang
📚 Learning: 2025-09-16T17:16:07.820Z
Learnt from: keivenchang
Repo: ai-dynamo/dynamo PR: 3051
File: container/templates/Dockerfile.vllm.j2:221-233
Timestamp: 2025-09-16T17:16:07.820Z
Learning: In the dynamo project, when converting Dockerfiles to Jinja2 templates, the primary goal is backward compatibility - generated Dockerfiles must be identical to the originals. Security improvements or other enhancements that would change the generated output are out of scope for templating PRs and should be addressed separately.

Applied to files:

  • container/Dockerfile.sglang
📚 Learning: 2025-08-30T20:43:10.091Z
Learnt from: keivenchang
Repo: ai-dynamo/dynamo PR: 2797
File: .devcontainer/devcontainer.json:12-12
Timestamp: 2025-08-30T20:43:10.091Z
Learning: In the dynamo project, devcontainer.json files use templated container names (like "dynamo-vllm-devcontainer") that are automatically processed by the copy_devcontainer.sh script to generate framework-specific configurations with unique names, preventing container name collisions.

Applied to files:

  • container/Dockerfile.sglang
📚 Learning: 2025-08-05T22:51:59.230Z
Learnt from: dmitry-tokarev-nv
Repo: ai-dynamo/dynamo PR: 2300
File: pyproject.toml:64-66
Timestamp: 2025-08-05T22:51:59.230Z
Learning: The ai-dynamo/dynamo project does not ship ARM64 wheels, so platform markers to restrict dependencies to x86_64 are not needed in pyproject.toml dependencies.

Applied to files:

  • container/Dockerfile.sglang
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (5)
  • GitHub Check: trtllm (arm64)
  • GitHub Check: operator (arm64)
  • GitHub Check: sglang (arm64)
  • GitHub Check: vllm (arm64)
  • GitHub Check: Build and Test - dynamo
🔇 Additional comments (2)
container/Dockerfile.sglang (1)

386-387: Correct placement and clean pattern.

The selective copying approach (lines 380-384) followed by explicit pyproject.toml placement ensures pytest markers are available while maintaining clarity about workspace dependencies.

container/Dockerfile.trtllm (1)

333-334: Clean, consistent pattern.

The selective copying approach (lines 326-331) followed by explicit pyproject.toml placement mirrors sglang and maintains consistency across framework-specific Dockerfiles. The pattern is efficient and clear.

…OK, we don't control nvidia-modelopt install and stick with default from trtllm install
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants