4 papers · 1 filter
SparkDiffusion: Mitigating the High-Sparsity Trap --- A Unified Framework for up to Single-GPU Acceleration of Visual Generation
Yuxi Liu, Haoyu Li, Zekun Zhang +12
Video diffusion transformers are expensive because attention dominates long spatiotemporal token sequences. We identify the \emph{high-sparsity trap}: at extreme attention sparsity…
RoLA: Rotary-Positioned Low-Rank Linear Attention for Efficient Diffusion Transformers
Zekun Zhang, Yixiang Cai, Yuxi Liu +9
Diffusion Transformers (DiTs) achieve strong video generation quality, but their dense spatiotemporal self-attention scales quadratically with sequence length and quickly becomes t…
RoPeSLR: 3D RoPE-driven Sparse-LowRank Attention for Efficient Diffusion Transformers
Yuxi Liu, Zekun Zhang, Yixiang Cai +3
Diffusion Transformers (DiTs) have revolutionized high-fidelity video generation, yet their attention complexity poses a formidable bottleneck for long-sequence…
Mixture of Distributions Matters: Dynamic Sparse Attention for Efficient Video Diffusion Transformers
Yuxi Liu, Yipeng Hu, Zekun Zhang +2
While Diffusion Transformers (DiTs) have achieved notable progress in video generation, this long-sequence generation task remains constrained by the quadratic complexity inherent…