8 papers
Universal YOCO for Efficient Depth Scaling
Yutao Sun, Li Dong, Tianzhu Ye +3
The rise of test-time scaling has remarkably boosted the reasoning and agentic proficiency of Large Language Models (LLMs). Yet, standard Transformers struggle to scale inference-t…
Geometric Autoencoder for Diffusion Models
Hangyu Liu, Jianyong Wang, Yutao Sun
Latent diffusion models have established a new state-of-the-art in high-resolution visual generation. Integrating Vision Foundation Model priors improves generative efficiency, yet…
LeanK: Learnable K Cache Channel Pruning for Efficient Decoding
Yike Zhang, Zhiyuan He, Huiqiang Jiang +4
Large language models (LLMs) enable long-context tasks but face efficiency challenges due to the growing key-value (KV) cache. We propose LeanK, a learning-based method that prunes…
Rectified Sparse Attention
Yutao Sun, Tianzhu Ye, Li Dong +6
Efficient long-sequence generation is a critical challenge for Large Language Models. While recent sparse decoding methods improve efficiency, they suffer from KV cache misalignmen…
FR-Spec: Accelerating Large-Vocabulary Language Models via Frequency-Ranked Speculative Sampling
Weilin Zhao, Tengyu Pan, Xu Han +9
Speculative sampling has emerged as an important technique for accelerating the auto-regressive generation process of large language models (LLMs) by utilizing a draft-then-verify…
FocusLLM: Precise Understanding of Long Context by Dynamic Condensing
Zhenyu Li, Yike Zhang, Tengyu Pan +6
Empowering LLMs with the ability to precisely understand long contexts is crucial for many downstream applications. However, handling long contexts with conventional transformer ar…