5 papers
Thinking as Compression: Your Reasoning Model is Secretly a Context Compressor
Guoxin Ma, Yibing Liu, Chengzhengxu Li +7
Context compression aims to shorten long context inputs with minimal information loss for LLM inference acceleration. While existing methods have shown promise, they typically rely…
ConsistRM: Improving Generative Reward Models via Consistency-Aware Self-Training
Yu Liang, Liangxin Liu, Longzheng Wang +5
Generative reward models (GRMs) have emerged as a promising approach for aligning Large Language Models (LLMs) with human preferences by offering greater representational capacity…
ReflectRM: Boosting Generative Reward Models via Self-Reflection within a Unified Judgment Framework
Kai Qin, Liangxin Liu, Yu Liang +7
Reward Models (RMs) are critical components in the Reinforcement Learning from Human Feedback (RLHF) pipeline, directly determining the alignment quality of Large Language Models (…
Advancing General-Purpose Reasoning Models with Modular Gradient Surgery
Min Cai, Yu Liang, Longzheng Wang +6
Reinforcement learning (RL) has played a central role in recent advances in large reasoning models (LRMs), yielding strong gains in verifiable and open-ended reasoning. However, tr…
TRE: Encouraging Exploration in the Trust Region
Chao Huang, Yujing Lu, Quangang Li +8
Entropy regularization is a standard technique in reinforcement learning (RL) to enhance exploration, yet it yields negligible effects or even degrades performance in Large Languag…