5 citations · 6 across the 19 of their papers we have counts for
6 papers · 1 filter
RL's Razor: Why Online Reinforcement Learning Forgets Less
Idan Shenfeld, Jyothish Pari, Pulkit Agrawal
Comparison of fine-tuning models with reinforcement learning (RL) and supervised fine-tuning (SFT) reveals that, despite similar performance at a new task, RL preserves prior knowl…
Best-of-N through the Smoothing Lens: KL Divergence and Regret Analysis
Gholamali Aminian, Idan Shenfeld, Amir R. Asadi +2
A simple yet effective method for inference-time alignment of generative models is Best-of- (BoN), where outcomes are sampled from a reference policy, evaluated using a prox…
Beyond Binary Rewards: Training LMs to Reason About Their Uncertainty
Mehul Damani, Isha Puri, Stewart Slocum +4
When language models (LMs) are trained via reinforcement learning (RL) to generate natural language "reasoning chains", their performance improves on a variety of difficult questio…
LLM Hypnosis: Exploiting User Feedback for Unauthorized Knowledge Injection to All Users
Almog Hilel, Riddhi Bhagwat, Idan Shenfeld +2
We describe a vulnerability in language models (LMs) trained with user feedback, whereby a single user can persistently alter LM knowledge and behavior given only the ability to pr…
Language Model Personalization via Reward Factorization
Idan Shenfeld, Felix Faltings, Pulkit Agrawal +1
Modern large language models (LLMs) are optimized for human-aligned responses using Reinforcement Learning from Human Feedback (RLHF). However, existing RLHF approaches assume a un…
KL-Regularized RLHF with Multiple Reference Models: Exact Solutions and Sample Complexity
Gholamali Aminian, Amir R. Asadi, Idan Shenfeld +1
Recent methods for aligning large language models (LLMs) with human feedback predominantly rely on a single reference model, which limits diversity, model overfitting, and underuti…