4 papers · 1 filter
REAR: Test-time Preference Realignment through Reward Decomposition
Fuxiang Zhang, Pengcheng Wang, Chenran Li +6
Aligning large language models (LLMs) with diverse user preferences is a critical yet challenging task. While post-training methods can adapt models to specific needs, they often r…
Generalist Reward Models: Found Inside Large Language Models
Yi-Chen Li, Tian Xu, Yang Yu +6
The alignment of Large Language Models (LLMs) is critically dependent on reward models trained on costly human preference data. While recent work explores bypassing this cost with…
Sentence-level Reward Model can Generalize Better for Aligning LLM from Human Preference
Wenjie Qiu, Yi-Chen Li, Xuqin Zhang +4
Learning reward models from human preference datasets and subsequently optimizing language models via reinforcement learning has emerged as a fundamental paradigm for aligning LLMs…
Controlling Large Language Model with Latent Actions
Chengxing Jia, Ziniu Li, Pengyuan Wang +4
Adapting Large Language Models (LLMs) to downstream tasks using Reinforcement Learning (RL) has proven to be an effective approach. However, LLMs do not inherently define the struc…