7 papers
ActiveUltraFeedback: Efficient Preference Data Generation using Active Learning
Davit Melikidze, Marian Schneider, Jessica Lam +4
Reinforcement Learning from Human Feedback (RLHF) has become the standard for aligning Large Language Models (LLMs), yet its efficacy is bottlenecked by the high cost of acquiring…
Scalable Ride-Sourcing Vehicle Rebalancing with Service Accessibility Guarantee: A Constrained Mean-Field Reinforcement Learning Approach
Matej Jusup, Kenan Zhang, Zhiyuan Hu +3
The expansion of ride-sourcing services such as Uber and Lyft has reshaped urban transportation by offering flexible, on-demand mobility via mobile applications. Despite convenienc…
RewardUQ: A Unified Framework for Uncertainty-Aware Reward Models
Daniel Yang, Samuel Stante, Florian Redhardt +5
Reward models are central to aligning large language models (LLMs) with human preferences. Yet most approaches rely on pointwise reward estimates that overlook the epistemic uncert…
Aligning Language Models from User Interactions
Thomas Kleine Buening, Jonas Hübotter, Barna Pásztor +3
Multi-turn user interactions are among the most abundant data produced by language models, yet we lack effective methods to learn from them. While typically discarded, these intera…
Stackelberg Learning from Human Feedback: Preference Optimization as a Sequential Game
Barna Pásztor, Thomas Kleine Buening, Andreas Krause
We introduce Stackelberg Learning from Human Feedback (SLHF), a new framework for preference optimization. SLHF frames the alignment problem as a sequential-move game between two p…
Bandits with Preference Feedback: A Stackelberg Game Perspective
Barna Pásztor, Parnian Kassraie, Andreas Krause
Bandits with preference feedback present a powerful tool for optimizing unknown target functions when only pairwise comparisons are allowed instead of direct value queries. This mo…