9 papers
DC-Leap: Training-Free Acceleration of dLLMs via Draft-Guided Contiguous Leaping Decoding
Yanhua Jiao, Tianyi Wu, Xiaoxi Sun +6
While parallel decoding is central to the efficiency of Diffusion Large Language Models (dLLMs), current strategies are often hindered by overly conservative confidence thresholds.…
PSD: Pushing the Pareto Frontier of Diffusion LLMs via Parallel Speculative Decoding
Shengyin Sun, Yiming Li, Renxi Liu +7
Diffusion large language models (dLLMs) generate text by iteratively denoising masked token sequences. Although dLLMs can predict all masked positions in parallel within each step,…
Kernel Foundry: A Diagnosis-driven Evolutionary Kernel Optimizer with Multi-Experts
Zixuan Huang, Da Chen, Kecheng Huang +5
Generating high-performance GPU kernels remains challenging due to the need for both correctness and hardware-aware optimization. While large language models (LLMs) show promise in…
Efficient Reasoning with Balanced Thinking
Yulin Li, Tengyao Tu, Li Ding +5
Large Reasoning Models (LRMs) have shown remarkable reasoning capabilities, yet they often suffer from overthinking, expending redundant computational steps on simple problems, or…
Why Attention Patterns Exist: A Unifying Temporal Perspective Analysis
Qingyue Yang, Jie Wang, Xing Li +6
Attention patterns play a crucial role in both training and inference of large language models (LLMs). Prior works have identified individual patterns such as retrieval heads, sink…
Attention-Aware GNN-based Input Defense against Multi-Turn LLM Jailbreak
Zixuan Huang, Kecheng Huang, Lihao Yin +4
Large Language Models (LLMs) have gained significant traction in various applications, yet their capabilities present risks for both constructive and malicious exploitation. Despit…