5 papers
AlignTune: Modular Toolkit for Post-Training Alignment of Large Language Models
R E Zera Marveen Lyngkhoi, Chirag Chawla, Pratinav Seth +5
Post-training alignment is central to deploying large language models (LLMs), yet practical workflows remain split across backend-specific tools and ad-hoc glue code, making experi…
Beyond Uniform Credit: Causal Credit Assignment for Policy Optimization
Mykola Khandoga, Rui Yuan, Vinay Kumar Sankarapu
Policy gradient methods for language model reasoning, such as GRPO and DAPO, assign uniform credit to all generated tokens - the filler phrase "Let me think" receives the same grad…
Beyond KL Divergence: Policy Optimization with Flexible Bregman Divergences for LLM Reasoning
Rui Yuan, Mykola Khandoga, Vinay Kumar Sankarapu
Policy optimization methods like Group Relative Policy Optimization (GRPO) and its variants have achieved strong results on mathematical reasoning and code generation tasks. Despit…
The Initialization Determines Whether In-Context Learning Is Gradient Descent
Shifeng Xie, Rui Yuan, Simone Rossi +1
In-context learning (ICL) in large language models (LLMs) is a striking phenomenon, yet its underlying mechanisms remain only partially understood. Previous work connects linear se…
From predictions to confidence intervals: an empirical study of conformal prediction methods for in-context learning
Zhe Huang, Simone Rossi, Rui Yuan +1
Transformers have become a standard architecture in machine learning, demonstrating strong in-context learning (ICL) abilities that allow them to learn from the prompt at inference…