4 papers · 1 filter
Rubric Dropout: A Simple Way to Mitigate Reward Hacking in Rubric-as-Reward RL
Minglai Yang, Xinyu Guo, Utkarsh Tyagi +6
Reinforcement learning against rubrics, lists of criteria graded by an LLM judge, has become a standard way to post-train language models on tasks with no deterministic answer. The…
EchoRL: Reinforcement Learning via Rollout Echoing
Jinhe Bi, Aniri, Minglai Yang +9
Reinforcement Learning with Verifiable Rewards is an effective route for post-training to strengthen the reasoning capability of large language models. However, as training proceed…
AlignSAE: Concept-Aligned Sparse Autoencoders
Minglai Yang, Xinyu Guo, Zhengliang Shi +4
Large Language Models (LLMs) encode factual knowledge within hidden parametric spaces that are difficult to inspect or control. While Sparse Autoencoders (SAEs) can decompose hidde…
Improving the Data-efficiency of Reinforcement Learning by Warm-starting with LLM
Thang Duong, Minglai Yang, Chicheng Zhang
We investigate the usage of Large Language Model (LLM) in collecting high-quality data to warm-start Reinforcement Learning (RL) algorithms for learning in some classical Markov De…