Showing cs.LGShow all
2 papers · 1 filter
cs.LG2026
Attention Sinks and Outliers in Attention Residuals
Haozheng Luo, Haoran Dai, Shaoyang Zhang +10
We propose OASIS, an outlier- and sink-aware technique built on inter-layer null signaling. As AttnResidual architectures introduce an additional depth-wise normalization channel,…
cs.LG2025
Learning to Rank Chain-of-Thought: Using a Small Model
Eric Hanchen Jiang, Haozheng Luo, Shengyuan Pang +9
Large Language Models (LLMs) struggle with reliable mathematical reasoning, and current verification methods are often computationally expensive. This paper introduces the Energy O…