Trust Region Masking for Long-Horizon LLM Reinforcement Learning

Yingru Li, Jiacai Liu, Jiawei Xu +4

Policy gradient methods for Large Language Models optimize a policy $\ddot{I}{€}_{\hat{I}}¸$ via a surrogate objective computed from samples of a rollout policy $\ddot{I}{€}_{\text{roll}}$. However, modern…