4 papers · 1 filter
MATCH: Modulating Attention via In-Context Retrieval for Long-Context Transformers
Linrui Ma, Chun Hei Lo, Xinyu Wang +12
The quadratic computational cost of traditional attention mechanisms poses a major bottleneck to the scalability and practical deployment of large language models (LLMs), particula…
Domino: Decoupling Causal Modeling from Autoregressive Drafting in Speculative Decoding
Jianuo Huang, Yaojie Zhang, Qituan Zhang +3
Speculative decoding accelerates LLM inference by drafting multiple tokens and verifying them in parallel with the target model. However, its practical speedup is constrained by th…
Diffusion LLM with Native Variable Generation Lengths: Let [EOS] Lead the Way
Yicun Yang, Cong Wang, Shaobo Wang +4
Diffusion-based large language models (dLLMs) have exhibited substantial potential for parallel text generation, which may enable more efficient generation compared to autoregressi…
Self Speculative Decoding for Diffusion Large Language Models
Yifeng Gao, Ziang Ji, Yuxuan Wang +3
Diffusion-based Large Language Models (dLLMs) have emerged as a competitive alternative to autoregressive models, offering unique advantages through bidirectional attention and par…