3 papers
cs.CL2026
A Unified View of Attention and Residual Sinks: Outlier-Driven Rescaling is Essential for Transformer Training
Zihan Qiu, Zeyu Huang, Kaiyue Wen +16
We investigate the functional role of emergent outliers in large language models, specifically attention sinks (a few tokens that consistently receive large attention logits) and r…
cs.CL2025
Teaching LLMs to Abstain via Fine-Grained Semantic Confidence Reward
Hao An, Yang Xu
Mitigating hallucinations in Large Language Models (LLMs) is critical for their reliable deployment. Existing methods typically fine-tune LLMs to abstain from answering questions b…
cs.SE2025
Towards Better Correctness and Efficiency in Code Generation
Yunlong Feng, Yang Xu, Xiao Xu +2
While code large language models have demonstrated remarkable progress in code generation, the generated code often exhibits poor runtime efficiency, limiting its practical applica…