Skip to content

feat: add Nemotron 3.5 S5000 fused-MoE config - #111

Draft
yeahdongcn wants to merge 17 commits into
mainfrom
xd/nemotron35-e128-n1856
Draft

feat: add Nemotron 3.5 S5000 fused-MoE config#111
yeahdongcn wants to merge 17 commits into
mainfrom
xd/nemotron35-e128-n1856

Conversation

@yeahdongcn

@yeahdongcn yeahdongcn commented Sep 9, 2026

Copy link
Copy Markdown
Collaborator

Summary

Make the torchada Triton fused-MoE tuning path safe for Nemotron 3.5 Lightning BF16 (E=128,N=1856, non-gated relu2_no_mul) on S5000. The branch removes the unvalidated auto-loaded JSON candidate, prevents stale routing metadata from crossing graph replay, rejects the non-graph-safe Python alignment fallback during capture, and fixes tuner input reuse and packed non-gated shapes.

Validation

  • tests/test_moe_pipeline.py tests/test_nemotron_moe_config.py: 14 passed.
  • Pre-commit hooks: passed.
  • The prior standalone graph result used a routing cache and is not retained as correctness evidence. A new GPU graph test must cover route mutation and compare each replay with an eager reference before adding a config.
  • An independent vLLM-MUSA target-only diagnostic on S5000 (driver 5.2.0, torch 2.11.0.post1+musa5.2.0, Triton 3.2.0) loaded the 61.31 GiB checkpoint and produced deterministic semantic output with graph mode NONE; this does not qualify the torchada kernel or a full graph service.

Pending gate

No Nemotron autotune JSON is currently shipped from this branch. Before merging, run the tuner on S5000 with native alignment, dynamic route graph replay, finite/reference output checks, and measured per-M bucket results. Then integrate only the candidates that pass those gates and re-run vLLM-MUSA with PCG and full graph enabled.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant