4 papers · 1 filter
PSD: Pushing the Pareto Frontier of Diffusion LLMs via Parallel Speculative Decoding
Shengyin Sun, Yiming Li, Renxi Liu +7
Diffusion large language models (dLLMs) generate text by iteratively denoising masked token sequences. Although dLLMs can predict all masked positions in parallel within each step,…
Why Attention Patterns Exist: A Unifying Temporal Perspective Analysis
Qingyue Yang, Jie Wang, Xing Li +6
Attention patterns play a crucial role in both training and inference of large language models (LLMs). Prior works have identified individual patterns such as retrieval heads, sink…
Preserving LLM Capabilities through Calibration Data Curation: From Analysis to Optimization
Bowei He, Lihao Yin, Huiling Zhen +5
Post-training compression has been a widely employed approach to scale down large language model (LLM) and facilitate efficient inference. In various proposed compression methods,…
Accelerating Large Language Model Reasoning via Speculative Search
Zhihai Wang, Jie Wang, Jilai Pan +5
Tree-search-based reasoning methods have significantly enhanced the reasoning capability of large language models (LLMs) by facilitating the exploration of multiple intermediate re…