6 papers
Sink-Sparse Hybrid Attention for Efficient Inference in T2I/T2V Diffusion Transformers
Bingxin Xing, Yu Zhang, Jiatong Ding +2
Diffusion Transformers (DiTs) have become a common backbone for high-quality text-to-image and text-to-video generation, but repeated dense self-attention remains a major inference…
HyQuant: Hybrid-Precision Quantization for LLM Attention
Jiatong Ding, Bingxin Xing, Yu Zhang +9
Quantization has been widely adopted in LLM training and inference to reduce cost and improve efficiency. However, low-bit quantization of the \emph{attention} module often introdu…
EntropyMoE: Entropy-Aware Sparse Expert Routing for Tokenizer-Free LLMs
Bo Liu, Muxuab Yu, Yu Zhang +2
Recent byte-level large language models (LLMs) have made tokenizer-free modeling increasingly competitive by grouping bytes into dynamically sized patches. However, existing byte-p…
S2O: Early Stopping for Sparse Attention via Online Permutation
Yu Zhang, Songwei Liu, Chenqian Yan +4
Attention scales quadratically with sequence length, fundamentally limiting long-context inference. Existing block-granularity sparsification can reduce latency, but coarse blocks…
AnchorAttention: Difference-Aware Sparse Attention with Stripe Granularity
Yu Zhang, Dong Guo, Fang Wu +3
Large Language Models (LLMs) with extended context lengths face significant computational challenges during the pre-filling phase, primarily due to the quadratic complexity of self…
ARFlow: Autoregressive Flow with Hybrid Linear Attention
Mude Hui, Rui-Jie Zhu, Songlin Yang +5
Flow models are effective at progressively generating realistic images, but they generally struggle to capture long-range dependencies during the generation process as they compres…