8 citations · 12 across the 11 of their papers we have counts for
11 papers
AlignTune: Modular Toolkit for Post-Training Alignment of Large Language Models
R E Zera Marveen Lyngkhoi, Chirag Chawla, Pratinav Seth +5
Post-training alignment is central to deploying large language models (LLMs), yet practical workflows remain split across backend-specific tools and ad-hoc glue code, making experi…
Beyond Uniform Credit: Causal Credit Assignment for Policy Optimization
Mykola Khandoga, Rui Yuan, Vinay Kumar Sankarapu
Policy gradient methods for language model reasoning, such as GRPO and DAPO, assign uniform credit to all generated tokens - the filler phrase "Let me think" receives the same grad…
Beyond KL Divergence: Policy Optimization with Flexible Bregman Divergences for LLM Reasoning
Rui Yuan, Mykola Khandoga, Vinay Kumar Sankarapu
Policy optimization methods like Group Relative Policy Optimization (GRPO) and its variants have achieved strong results on mathematical reasoning and code generation tasks. Despit…
The Initialization Determines Whether In-Context Learning Is Gradient Descent
Shifeng Xie, Rui Yuan, Simone Rossi +1
In-context learning (ICL) in large language models (LLMs) is a striking phenomenon, yet its underlying mechanisms remain only partially understood. Previous work connects linear se…
From predictions to confidence intervals: an empirical study of conformal prediction methods for in-context learning
Zhe Huang, Simone Rossi, Rui Yuan +1
Transformers have become a standard architecture in machine learning, demonstrating strong in-context learning (ICL) abilities that allow them to learn from the prompt at inference…
Enhancing Policy Gradient with the Polyak Step-Size Adaption
Yunxiang Li, Rui Yuan, Chen Fan +4
Policy gradient is a widely utilized and foundational algorithm in the field of reinforcement learning (RL). Renowned for its convergence guarantees and stability compared to other…
A Novel Framework for Policy Mirror Descent with General Parameterization and Linear Convergence
Carlo Alfano, Rui Yuan, Patrick Rebeschini
Modern policy optimization methods in reinforcement learning, such as TRPO and PPO, owe their success to the use of parameterized policies. However, while theoretical guarantees ha…
Linear Convergence of Natural Policy Gradient Methods with Log-Linear Policies
Rui Yuan, Simon S. Du, Robert M. Gower +2
We consider infinite-horizon discounted Markov decision processes and study the convergence rates of the natural policy gradient (NPG) and the Q-NPG methods with the log-linear pol…
A general sample complexity analysis of vanilla policy gradient
Rui Yuan, Robert M. Gower, Alessandro Lazaric
We adapt recent tools developed for the analysis of Stochastic Gradient Descent (SGD) in non-convex optimization to obtain convergence and sample complexity guarantees for the vani…
SAN: Stochastic Average Newton Algorithm for Minimizing Finite Sums
Jiabin Chen, Rui Yuan, Guillaume Garrigos +1
We present a principled approach for designing stochastic Newton methods for solving finite sum optimization problems. Our approach has two steps. First, we re-write the stationari…
Sketched Newton-Raphson
Rui Yuan, Alessandro Lazaric, Robert M. Gower
We propose a new globally convergent stochastic second order method. Our starting point is the development of a new Sketched Newton-Raphson (SNR) method for solving large scale non…