Repository navigation
Conversation
This was referenced Sep 1, 2026
Collaborator
|
Please add the tutorial document on how to run with Ascend NPU. |
The VAE weight-side conversion to channels_last_3d is already guarded by cuDNN availability, but the activation-side casts in CausalConv3d.forward and the spatial-parallel halo conv were unconditional. NPU and CPU reject channels_last_3d activations, so Wan2.2 VAE forward failed on Ascend with ERR01007 ("NPU contiguous operator only supported contiguous memory format"). Route non-CUDA tensors through standard contiguous instead.
Verified: Wan2.2-TI2V-5B text-to-video smoke on Ascend 910B2 (CANN 8.2, torch_npu 2.9.0) completes end to end; CUDA path unchanged.
create_device_mesh_from_config defaulted device_type to "cuda" and 12 of 14 call sites rely on the default, so parallel denoising on NPU crashed inside torch DeviceMesh with "integer modulo by zero" (zero visible CUDA devices). Resolve the default from current_platform; explicit arguments keep working. Verified: 4-card (cfg=2 x ulysses=2) Wan2.2-TI2V-5B run on Ascend 910B2 builds the [cfg, ulysses] mesh over hccl; on CUDA the default resolves to "cuda" as before.
Parallel workers exchange request and result tensors over torch.multiprocessing queues, which relies on device IPC. torch_npu cross-process sharing raised "devptr INTERNAL ASSERT FAILED ... entry in cache has missing shared_ptr" when rebuilding queued NPU tensors. Enable the existing queue_with_cpu marshalling by default on non-CUDA platforms, mirror it on the result path, and move results back to the stage device in the main process. Worker dispatch already moves inputs to the local device, so the contract is unchanged and CUDA keeps direct device-queue transport. Verified: 4-card Wan2.2-TI2V-5B denoising on Ascend 910B2 completes with workers exiting cleanly.
ops/fp8_attention.py defined @triton.jit kernels at module scope, making "import triton" unconditional for every consumer of the wan_video and minimax DiT import chains even though the fused path is CUDA-only by contract. Move the two kernels and their launcher to telefuser/kernel/triton/fp8_attention.py and import them lazily inside quantize_fp8_qkv after its existing CUDA validation, matching the established ops -> kernel.triton dispatch pattern (see ops/rotary.py and kernel/__init__.py). The pure-torch quantize/dequantize helpers are unchanged. Verified: wan22 pipeline import and Wan2.2-TI2V-5B smoke succeed on an NPU host without triton installed; tests/unit/models/test_wan_video_sol_attention.py passes with CUDA-only cases skipped.
recv, recv_latent, and recv_latent_async allocated receive buffers with device="cuda", which fails on NPU-only hosts before irecv can run. Allocate on current_platform.device_type (identical behavior on CUDA) and parameterize the test tensor device the same way so the suite runs on CUDA, NPU, and CPU hosts. Verified: tests/unit/distributed/test_pp_comm.py 12 passed on Ascend 910B2 (previously 6 device-related failures).
The non-CUDA result path introduced for CPU-marshalled queues moved results back to self._stage.device inside _wait_result. That breaks worker unit tests on non-CUDA hosts (mocked stages resolve .device through the method proxy or to a MagicMock), and it is unnecessary: stage entry points already place their inputs, and the Wan VAE decode paths move latents themselves. Drop the move so _wait_result returns results untouched; workers still marshal results through CPU when device IPC is unavailable. Verified: tests/unit/worker passes on an Ascend host (same non-CUDA branch as CPU CI) and the 4-card Wan2.2-TI2V-5B smoke still completes; ruff clean.
The example hardcoded device="cuda", so running it on an NPU host required editing the script. Default the pipeline device to current_platform.device_type so the same command runs unmodified on CUDA and NPU; on CUDA hosts this resolves to "cuda" as before. Verified: python examples/wan_video/wan22_t2v_5b.py --gpu_num 1 and --gpu_num 4 run unmodified on Ascend 910B (50 steps, 121 frames, default prompt).
Wan22VideoVAE.decode's parallel branch called torch.stack on its input, assuming a list of per-video tensors, but VAEStage.decode_video passes a batched [B, C, T, H, W] tensor; the serial branch only worked because iterating a tensor yields its batch slices. Any enable_vae_parallel run of the Wan2.2 48-channel VAE therefore failed with TypeError regardless of platform. Normalize tensor inputs before stacking. Found while enabling spatially parallel VAE decode on Ascend 910B2; the parallel path now proceeds to communicator setup (further multi-communicator progress on that host is limited by its CANN driver, see branch notes).
Concurrent worker groups sharing the same NPU devices (e.g. denoising plus VAE workers) collide on HCCL's default data-plane socket range and fail comm init with EJ0003 ("IP address and port have been bound already"). Allocate a distinct HCCL_IF_BASE_PORT per spawned group on NPU platforms, mirroring the existing per-group MASTER_PORT allocation; other platforms are untouched.
Verified: single-group 4-card Wan2.2-TI2V-5B regression on Ascend 910B2 passes with the env applied (33.4s generate, parity with baseline).
…o guides Mirror the ROCm registration pattern for the NPU support validated by this branch, updating the guides introduced on main: - docs/en|zh/platforms.md: expand the "NPU and CPU" section with the native-fallback attention path (TORCH_SDPA, no triton/tf-kernel needed), forward_npu dispatch, HCCL with CPU-marshalled worker queues and per-group HCCL_IF_BASE_PORT isolation, and the validated wan22_t2v_5b.py entry point on Atlas 910B (CANN 8.2); note that Wan2.2 A14B is not yet exercised. - docs/en|zh/installation.md: add an Ascend NPU requirements row and a verification note (torch_npu selection, ASCEND_RT_VISIBLE_DEVICES, torch.cuda.is_available() False is expected). - examples/wan_video/README.md: add the Ascend requirements bullet and platform auto-detection / validated-hardware bullets to wan22_t2v_5b.py. Verification: python scripts/docs/prepare_cookbook.py build passes with rendered pages, assets, and anchors validated; the tests/docs unittest failures on this host are macOS /var tempdir symlink artifacts unrelated to these files. English additions stay within the 120-column style.
yx0716
force-pushed
the
feat/npu-adapt
branch
from
September 11, 2026 08:03
8a4220e to
54a2b94
Compare
Restore the mainline smoothing, scale and correction paths for Wan and MiniMax when the optional Triton implementation is split out. The CPU paths import without Triton. All 18 FP8 function definitions match main apart from import placement. Four CPU smoothing modes matched bitwise. H100 execution remains a separate qualification.
Bind visible accelerator IDs before initializing the process group. Stage NPU tensors on the host for both multiprocessing transports, reserve and recycle aligned port blocks, and clean up a partially spawned HCCL group. Restore the CPU and GPU Conv3d activation layout while using NCDHW on Ascend. Exercise independent-port owners, shutdown, source metadata and uneven shards on CPU.
Pin the 910B CANN 9.3.0-beta.1 arm64 manifest, CPU PyTorch 2.11.0 and the Ascend 2.11.0.post4 wheel, including hashes for the runtime closure. Move GPU-specific packages into the cuda extra. Document source builds, release digests, device mapping and the 1/2/4-card offline T2V recipe in both languages. The documentation build and shell examples were checked. Hardware qualification is reported per card count.
CANN exports development tools through PYTHONPATH, including an optional profiler with absent dependencies and a separate NumPy constraint. Resolve and check the application venv with Python isolated mode while preserving the CANN entrypoint and its runtime library paths. Keep source labels after the locked dependency layer for cached application rebuilds.
Remove the stale Triton exclusions for the shared device mesh and pipeline communication suites. Exercise both list and batched Wan2.2 VAE inputs with tiled and spatial parallel dispatch. Validated the focused CPU suite without Triton: 141 passed, 8 skipped. Ruff and formatting checks passed.
Import the sparse SageAttention kernels only when that backend executes. LongCat and FlashVSR can now load their normal service entry points without the optional CUDA extra. Keep the same Triton functions and arguments for sparse attention. Skip the MiniMax Ulysses CUDA kernel tests when Triton itself is absent. Other missing dependencies still fail collection. Verified the CPU service parity tests and both model imports without Triton.
Export the locked Python environment and Wan example as an arm64 application image. Assemble it with the official CANN image and Ubuntu FFmpeg on the user host. Document the public GitHub Release download, checksum verification and Docker commands in both runbooks. Verified the original source-build path, the application archive and no-context assembly, the TeleFuser and Wan CLI entry points, a TorchCodec and FFmpeg round trip, both docs builds, and Bash syntax.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Run Wan2.2-TI2V-5B text-to-video on Atlas 910B2 with CANN 9.3.0-beta.1. Ascend uses TeleFuser’s existing platform and operator interfaces. Download the prebuilt arm64 Docker package or build from source. Docker combines the published TeleFuser application layer with the pinned official CANN image and Ubuntu FFmpeg. Public Docker package · Dockerfile · English guide · 中文手册
CUDA users who need Triton, torchao or bitsandbytes now install
telefuser[cuda]. ROCm users need the Triton build paired with their PyTorch distribution.Shared runtime changes
torch.stackerror for all callers, including CUDA/H100.mainbyte-for-byte.Validation
c27fabd). Wan took 157.07 s on one card and 92.05 s on two cards with CFG 2. The 121-frame 480p MP4s were byte-identical. Two-rank HCCL and tensor transfer also passed.a9a3495. Four-card CANN 9.3, distributed VAE output parity and H100/ROCm hardware remain unverified. CI checks