3 papers
cs.CV2026
Training-free sparse attention based on cumulative energy filtering
Chunlu Li, Yixuan Pan, Bai Du +5
Sparse attention accelerates Diffusion Transformers (DiTs) for video generation by computing only the important tokens while skipping the rest. The token selection strategy is key…
cs.LG2026
VFA: Relieving Vector Operations in Flash Attention with Global Maximum Pre-computation
Yupeng Sun, Yanzhao Li, Zhiqiang Zou +5
FlashAttention-style online softmax enables exact attention computation with linear memory by streaming score tiles through on-chip memory and maintaining a running maximum and nor…
cs.LG2026
OSC: Hardware Efficient W4A4 Quantization via Outlier Separation in Channel Dimension
Zhiyuan Zhang, Yanzhao Li, Zhiqiang Zou +4
While 4-bit quantization is essential for high-throughput deployment of Large Language Models, activation outliers often lead to significant accuracy degradation due to the restric…