Showing cs.LGShow all
3 papers · 1 filter
cs.LG2026
SHAPE: Stage-aware Hierarchical Advantage via Potential Estimation for LLM Reasoning
Zhengyang Ai, Zikang Shan, Xiaodong Ai +3
Process supervision has emerged as a promising approach for enhancing LLM reasoning, yet existing methods fail to distinguish meaningful progress from mere verbosity, leading to li…
cs.LG2026
BAPS: A Fine-Grained Low-Precision Scheme for Softmax in Attention via Block-Aware Precision reScaling
Zisheng Ye, Xiaoyu He, Maoyuan Song +10
As the performance gains from accelerating quantized matrix multiplication plateau, the softmax operation becomes the critical bottleneck in Transformer inference. This bottleneck…
cs.LG2026
A Provable Expressiveness Hierarchy in Hybrid Linear-Full Attention
Xiaowei Ye, Xiaoyu He, Chao Liao +2
Transformers serve as the foundation of most modern large language models. To mitigate the quadratic complexity of standard full attention, various efficient attention mechanisms,…