6 papers
Scaling Large Reasoning Models beyond Human Supervision: A Path toward Superintelligence
Zhiqin Yang, Jingwen Fu, Yuhan Liu +16
Recent advances in large reasoning models (LRMs) have shown that reinforcement learning with verifiable rewards (RLVR) can substantially improve reasoning in mathematics and code,…
A Survey of Audio Reasoning in Multimodal Foundation Models
Zhihan Guo, Wenqian Cui, Guan-Ting Lin +8
Reasoning has become a defining capability of modern foundation models, yet its development in the audio modality remains limited. Audio poses challenges that are distinct from tho…
Mathesis: Towards Formal Theorem Proving from Natural Languages
Yu Xuejun, Jianyuan Zhong, Zijin Feng +17
Recent advances in large language models show strong promise for formal reasoning. However, most LLM-based theorem provers have long been constrained by the need for expert-written…
Self-Adjust Softmax
Chuanyang Zheng, Yihang Gao, Guoxuan Chen +7
The softmax function is crucial in Transformer attention, which normalizes each row of the attention scores with summation to one, achieving superior performances over other altern…
DAPE V2: Process Attention Score as Feature Map for Length Extrapolation
Chuanyang Zheng, Yihang Gao, Han Shi +9
The attention mechanism is a fundamental component of the Transformer model, contributing to interactions among distinct tokens, in contrast to earlier feed-forward neural networks…
DAPE: Data-Adaptive Positional Encoding for Length Extrapolation
Chuanyang Zheng, Yihang Gao, Han Shi +8
Positional encoding plays a crucial role in transformers, significantly impacting model performance and length generalization. Prior research has introduced absolute positional enc…