4 papers
SparkDiffusion: Mitigating the High-Sparsity Trap --- A Unified Framework for up to Single-GPU Acceleration of Visual Generation
Yuxi Liu, Haoyu Li, Zekun Zhang +12
Video diffusion transformers are expensive because attention dominates long spatiotemporal token sequences. We identify the \emph{high-sparsity trap}: at extreme attention sparsity…
RoLA: Rotary-Positioned Low-Rank Linear Attention for Efficient Diffusion Transformers
Zekun Zhang, Yixiang Cai, Yuxi Liu +9
Diffusion Transformers (DiTs) achieve strong video generation quality, but their dense spatiotemporal self-attention scales quadratically with sequence length and quickly becomes t…
BADiff: Bandwidth Adaptive Diffusion Model
Xi Zhang, Hanwei Zhu, Yan Zhong +2
In this work, we propose a novel framework to enable diffusion models to adapt their generation quality based on real-time network bandwidth constraints. Traditional diffusion mode…
Learning Grouped Lattice Vector Quantizers for Low-Bit LLM Compression
Xi Zhang, Xiaolin Wu, Jiamang Wang +1
Large Language Models (LLMs) have demonstrated remarkable capabilities but typically require extensive computational resources and memory for inference. Post-training quantization…