8 citations · 12 across the 11 of their papers we have counts for
5 papers · 1 filter
Beyond KL Divergence: Policy Optimization with Flexible Bregman Divergences for LLM Reasoning
Rui Yuan, Mykola Khandoga, Vinay Kumar Sankarapu
Policy optimization methods like Group Relative Policy Optimization (GRPO) and its variants have achieved strong results on mathematical reasoning and code generation tasks. Despit…
The Initialization Determines Whether In-Context Learning Is Gradient Descent
Shifeng Xie, Rui Yuan, Simone Rossi +1
In-context learning (ICL) in large language models (LLMs) is a striking phenomenon, yet its underlying mechanisms remain only partially understood. Previous work connects linear se…
Enhancing Policy Gradient with the Polyak Step-Size Adaption
Yunxiang Li, Rui Yuan, Chen Fan +4
Policy gradient is a widely utilized and foundational algorithm in the field of reinforcement learning (RL). Renowned for its convergence guarantees and stability compared to other…
Linear Convergence of Natural Policy Gradient Methods with Log-Linear Policies
Rui Yuan, Simon S. Du, Robert M. Gower +2
We consider infinite-horizon discounted Markov decision processes and study the convergence rates of the natural policy gradient (NPG) and the Q-NPG methods with the log-linear pol…
A general sample complexity analysis of vanilla policy gradient
Rui Yuan, Robert M. Gower, Alessandro Lazaric
We adapt recent tools developed for the analysis of Stochastic Gradient Descent (SGD) in non-convex optimization to obtain convergence and sample complexity guarantees for the vani…