6 papers
When Less is Enough: Efficient Inference via Collaborative Reasoning
Yilei Chen, Sharut Gupta, Yannis Paschalidis +2
In this work, we introduce DUET (Dual-model Efficient Two-stage inference), a collaborative inference framework in which a capable model and a lightweight model work together to so…
Post-training Large Language Models for Diverse High-Quality Responses
Yilei Chen, Souradip Chakraborty, Lorenz Wolf +2
Reinforcement learning (RL) has emerged as a popular method for post-training large language models (LLMs). While improving the model's performance on downstream tasks, it often re…
Scaling In-Context Online Learning Capability of LLMs via Cross-Episode Meta-RL
Xiaofeng Lin, Sirou Zhu, Yilei Chen +6
Large language models (LLMs) achieve strong performance when all task-relevant information is available upfront, as in static prediction and instruction-following problems. However…
Active Preference Optimization for Sample Efficient RLHF
Nirjhar Das, Souradip Chakraborty, Aldo Pacchiano +1
Large Language Models (LLMs) aligned using Reinforcement Learning from Human Feedback (RLHF) have shown remarkable generation abilities in numerous tasks. However, collecting high-…
Language Model Personalization via Reward Factorization
Idan Shenfeld, Felix Faltings, Pulkit Agrawal +1
Modern large language models (LLMs) are optimized for human-aligned responses using Reinforcement Learning from Human Feedback (RLHF). However, existing RLHF approaches assume a un…
A Theoretical Framework for Partially Observed Reward-States in RLHF
Chinmaya Kausik, Mirco Mutti, Aldo Pacchiano +1
The growing deployment of reinforcement learning from human feedback (RLHF) calls for a deeper theoretical investigation of its underlying models. The prevalent models of RLHF do n…