Skip to content

Add PAN2 support and benchmark LightX2V vs TeleFuser on H3 - #55

Open
Uxito-Ada wants to merge 6 commits into
Tele-AI:mainfrom
Uxito-Ada:feat/pan2-lightx2v
Open

Uxito-Ada wants to merge 6 commits into
Tele-AI:mainfrom
Uxito-Ada:feat/pan2-lightx2v

Conversation

@Uxito-Ada

@Uxito-Ada Uxito-Ada commented Nov 20, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

  • Add native PAN2 T2V/I2V support through TeleFuser's existing pipeline, module loading, SP, TP, FP8 GEMM and Sol-Attention abstractions.
  • Add official-style PAN2 examples and documentation as well as and distributed/offload coverage.
  • Support Lightx2v-H3: document and validate the MiniMax-H3 LightX2V support and compare performance with TeleFuser's existing H3.

Performance and Quality

All measurements below are real checkpoint runs on 4× NVIDIA H100 80 GB. Timings are end-to-end generation wall time unless noted otherwise.

LightX2V-H3 vs TeleFuser-H3

  • Workload: 1344×768, 124 frames, 50 configured sampling points, TP2 × Ulysses SP2.
Implementation Generation Denoising Peak memory
Official LightX2V BF16 + SageAttention2 137.760 s 129.221 s 72,916 MiB
TeleFuser BF16 78.506 s 75.472 s 50,907 MiB
TeleFuser Q-SPA(FP8 + Sol-Attn) 52.275 s 49.277 s 43,501 MiB

TeleFuser BF16 is 1.75× faster than the matched official run and uses 30.2% less peak memory. TeleFuser Q-SPA(FP8 + Sol-Attn) is 2.64× faster and uses 40.3% less peak memory.

xychart-beta
    title "H3 generation time Comparison (lower is better)"
    x-axis [Official, TF-BF16, TF-QSPA]
    y-axis "seconds" 0 --> 150
    bar [137.760, 78.506, 52.275]
Loading
xychart-beta
    title "H3 peak memory Comparison (lower is better)"
    x-axis [Official, TF BF16, TF-QSPA]
    y-axis "MiB" 0 --> 80000
    bar [72916, 50907, 43501]
Loading

PAN2 with TeleFuser

  • Workload: official PAN2 checkpoint and sampling, 704×1248, 121 frames unless stated, 4× H100 80 GB. Q-SPA is applied by TeleFuser to enable hybrid SP+TP, FP8 Sol attention.

Ablation:

Configuration Generation Output
BF16 360.61 s 121×704×1248×3
SP2 226.28 s 121×704×1248×3
TP2 215.88 s 121×704×1248×3
FP8 + Sol-Attn 304.21 s 121×704×1248×3
SP2 + FP8 + Sol-Attn 218.66 s 121×704×1248×3
TP2 + FP8 + Sol-Attn 206.64 s 121×704×1248×3
I2V + TP2 + FP8 + Sol-Attn 191.86 s 121×704×1248×3
10 s + TP2 + FP8 + Sol-Attn 338.50 s 241×704×1248×3
xychart-beta
    title "PAN2 generation time by TeleFuser configuration"
    x-axis [BF16, SP2, TP2, FP8-Sol, SP2-FP8, TP2-FP8, I2V-TP2, 10s-TP2]
    y-axis "seconds" 0 --> 400
    bar [360.61, 226.28, 215.88, 304.21, 218.66, 206.64, 191.86, 338.50]
Loading

The PAN2 generates the 10-second, 241 frames, video. This has similar inputs and checkpoint to official PAN2 example.

TeleFuser PAN2 generated outputs

PAN2 TP2 FP8 Sol output

pan2-tp2-fp8-sol.mp4

TP2 + FP8 + Sol-Attn, 121 frames.

PAN2 10s output

pan2-10s-tp2-fp8-sol.mp4

10 s + TP2 + FP8 + Sol-Attn, 241 frames.

Tests

  • pytest -q tests/unit/models/test_pan2_dit.py tests/unit/models/test_h3_image_pan2_conditioning.py tests/unit/pipelines/test_h3_image_pan2_sampling.py
  • Ruff check and format checks
  • Documentation site link validation
  • Existing CPU integration, server, and Python 3.10/3.11/3.12 unit-test matrices

Architecture support

The MiniMax-H3 LightX2V path is documented as already supported and is covered by the matched benchmark above.

PAN2 uses TeleFuser's BaseModel abstraction, explicit conditional/unconditional CFG forwards, FP32 latent/noise accumulation, SP/TP process groups, staged module loading, and optional FP8/Sol-Attention execution.

Add native H3-Image six-head PDD generation and reference editing, plus PAN2
text-to-video and first-frame conditioning through existing pipeline, stage,
module-loading and worker contracts. Preserve checkpoint sampling order,
conditioning layouts, FP32 modulation and rotary buffers, and output conversion.

Reuse H3 parallel attention and implement PAN2 Ulysses/TP with padding removal,
FP8 linear quantization and Sol-Attn. Support repeated model CPU offload with
sharded weights. Keep H3 FP8 prefix/correction inputs contiguous at the existing
operator call sites. Add no shared framework configuration or service schema.

Provide runnable examples, source and checkpoint provenance, focused numerical
regressions, and reproducible distributed/worker validation tools. Twenty new
unit tests and 41 existing H3/FP8 regressions pass. Tiny SP2, TP2 and SP2xTP2
checks pass with FP8 GEMM, FP8 Sol-Attn and repeated CPU offload. Released
conditioner and image VAE outputs match independent references exactly.

Released-transformer generation verification remains in progress while its
weights download; the validation document records this boundary explicitly.
Preserve FP32 values only during dtype conversion. Device-only moves retain dtype directly and avoid duplicating the full modulation weights on GPU during CPU offload. Validated with four PAN2 model tests; rerun released-checkpoint generation after the observed offload OOM.
@Uxito-Ada Uxito-Ada changed the title Add PAN2 support and document LightX2V-H3 benchmarks Add PAN2 support and benchmark H3 with LightX2V Nov 20, 2026
@Uxito-Ada
Uxito-Ada requested a review from lzx1413 November 20, 2026 06:34
@Uxito-Ada Uxito-Ada changed the title Add PAN2 support and benchmark H3 with LightX2V Add PAN2 support and benchmark LightX2V vs TeleFuser on H3 Nov 20, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant