9 papers
DC-Leap: Training-Free Acceleration of dLLMs via Draft-Guided Contiguous Leaping Decoding
Yanhua Jiao, Tianyi Wu, Xiaoxi Sun +6
While parallel decoding is central to the efficiency of Diffusion Large Language Models (dLLMs), current strategies are often hindered by overly conservative confidence thresholds.…
PSD: Pushing the Pareto Frontier of Diffusion LLMs via Parallel Speculative Decoding
Shengyin Sun, Yiming Li, Renxi Liu +7
Diffusion large language models (dLLMs) generate text by iteratively denoising masked token sequences. Although dLLMs can predict all masked positions in parallel within each step,…
Kernel Foundry: A Diagnosis-driven Evolutionary Kernel Optimizer with Multi-Experts
Zixuan Huang, Da Chen, Kecheng Huang +5
Generating high-performance GPU kernels remains challenging due to the need for both correctness and hardware-aware optimization. While large language models (LLMs) show promise in…
Efficient Reasoning with Balanced Thinking
Yulin Li, Tengyao Tu, Li Ding +5
Large Reasoning Models (LRMs) have shown remarkable reasoning capabilities, yet they often suffer from overthinking, expending redundant computational steps on simple problems, or…
Why Attention Patterns Exist: A Unifying Temporal Perspective Analysis
Qingyue Yang, Jie Wang, Xing Li +6
Attention patterns play a crucial role in both training and inference of large language models (LLMs). Prior works have identified individual patterns such as retrieval heads, sink…
Preserving LLM Capabilities through Calibration Data Curation: From Analysis to Optimization
Bowei He, Lihao Yin, Huiling Zhen +5
Post-training compression has been a widely employed approach to scale down large language model (LLM) and facilitate efficient inference. In various proposed compression methods,…
Attention-Aware GNN-based Input Defense against Multi-Turn LLM Jailbreak
Zixuan Huang, Kecheng Huang, Lihao Yin +4
Large Language Models (LLMs) have gained significant traction in various applications, yet their capabilities present risks for both constructive and malicious exploitation. Despit…
HyperTree Planning: Enhancing LLM Reasoning via Hierarchical Thinking
Runquan Gui, Zhihai Wang, Jie Wang +7
Recent advancements have significantly enhanced the performance of large language models (LLMs) in tackling complex reasoning tasks, achieving notable success in domains like mathe…
Accelerating Large Language Model Reasoning via Speculative Search
Zhihai Wang, Jie Wang, Jilai Pan +5
Tree-search-based reasoning methods have significantly enhanced the reasoning capability of large language models (LLMs) by facilitating the exploration of multiple intermediate re…