Skip to content

[tutorial] feat: add MoE notebook example#3890

Open
karinseve wants to merge 1 commit into
NVIDIA-NeMo:mainfrom
karinseve:feat-moe-playbook
Open

[tutorial] feat: add MoE notebook example#3890
karinseve wants to merge 1 commit into
NVIDIA-NeMo:mainfrom
karinseve:feat-moe-playbook

Conversation

@karinseve
Copy link
Copy Markdown

What does this PR do ?

Add a self‑contained MoE (Mixture of Experts) training playbook

Changelog

Add examples/moe/ directory with a standalone MoE training example.

Provide a Jupyter notebook (moe.ipynb) walking through data setup, configuration, training, and basic evaluation for MoE models.

Include example configuration and launch commands tailored to Megatron‑Bridge MoE training.

Add brief inline documentation and comments to guide users adapting the playbook to their own setups.

GitHub Actions CI

See the CI section in the Contributing doc for how to trigger the CI. A Nvidia developer will need to approve and trigger the CI for external contributors.

Before your PR is "Ready for review"

Pre checks:

  • [ x ] Make sure you read and followed Contributor guidelines
  • [ x ] Did you write any new necessary tests? Not applicable if this is a documentation / example‑only change.
  • [ x ] Did you add or update any necessary documentation? Example notebook and in‑notebook explanations were added under examples/moe/
  • [ x ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc) N/A
    • Reviewer: Does the PR have correct import guards for all optional libraries?

If you haven't finished some of the above items you can still open "Draft" PR.

Additional Information

  • Related to # (issue)

Signed-off-by: Karin Sevegnani <ksevegnani@nvidia.com>
@copy-pr-bot
Copy link
Copy Markdown

copy-pr-bot Bot commented May 19, 2026

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@yaoyu-33 yaoyu-33 added area:recipe Training recipes and launch configs community-request docs Documentation-only updates or documentation debt docs-only With great power comes great responsibility. waiting-on-maintainers Waiting on maintainers to respond labels May 19, 2026
@yaoyu-33
Copy link
Copy Markdown
Contributor

@karinseve thanks for adding a playbook. Can you move to tutorials/ folder since it's notebook.

@yaoyu-33 yaoyu-33 added community-request waiting-on-customer Waiting on the original author to respond and removed waiting-on-maintainers Waiting on maintainers to respond labels May 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:recipe Training recipes and launch configs community-request docs Documentation-only updates or documentation debt docs-only With great power comes great responsibility. waiting-on-customer Waiting on the original author to respond

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants