18 citations · 34 across the 10 of their papers we have counts for
11 papers
Specifying Reward Functions for RL Without Environment Sampling
Stephane Hatgis-Kessell, W. Bradley Knox, Emma Brunskill
Enabling human stakeholders to specify reward functions that lead to their desired outcomes is a key challenge in deploying reinforcement learning agents. Preference-based methods…
A Framework for Designing Reward Functions: From Objectives to Features to Human-Aligned Reward Functions
Di Yang Shi, W. Bradley Knox
We present a formal process to enable non-experts to instantiate and iterate on human-aligned reward functions, i.e. reward functions that adhere to a given preference ordering ove…
GUIDE: Reinforcement Learning for Behavioral Action Support in Type 1 Diabetes
Saman Khamesian, Sri Harini Balaji, Di Yang Shi +5
Type 1 diabetes (T1D) management requires continuous adjustment of insulin and lifestyle behaviors to maintain blood glucose within a safe target range. Although automated insulin…
Harmful Traits of AI Companions
W. Bradley Knox, Katie Bradford, Samanta Varela Castro +6
Amid the growing prevalence of human-AI interaction, large language models and other AI-based entities increasingly provide forms of companionship to human users. Such AI companion…
Influencing Humans to Conform to Preference Models for RLHF
Stephane Hatgis-Kessell, W. Bradley Knox, Serena Booth +1
Designing a reinforcement learning from human feedback (RLHF) algorithm to approximate a human's unobservable reward function requires assuming, implicitly or explicitly, a model o…
Scaling Laws for Reward Model Overoptimization in Direct Alignment Algorithms
Rafael Rafailov, Yaswanth Chittepu, Ryan Park +5
Reinforcement Learning from Human Feedback (RLHF) has been crucial to the recent success of Large Language Models (LLMs), however, it is often a complex and brittle process. In the…
Learning Optimal Advantage from Preferences and Mistaking it for Reward
W. Bradley Knox, Stephane Hatgis-Kessell, Sigurdur Orn Adalgeirsson +4
We consider algorithms for learning reward functions from human preferences over pairs of trajectory segments, as used in reinforcement learning from human feedback (RLHF). Most re…
Contrastive Preference Learning: Learning from Human Feedback without RL
Joey Hejna, Rafael Rafailov, Harshit Sikchi +4
Reinforcement Learning from Human Feedback (RLHF) has emerged as a popular paradigm for aligning models with human intent. Typically RLHF algorithms operate in two phases: first, u…
Models of human preference for learning reward functions
W. Bradley Knox, Stephane Hatgis-Kessell, Serena Booth +3
The utility of reinforcement learning is limited by the alignment of reward functions with the interests of human stakeholders. One promising method for alignment is to learn the r…
Reward (Mis)design for Autonomous Driving
W. Bradley Knox, Alessandro Allievi, Holger Banzhaf +2
This article considers the problem of diagnosing certain common errors in reward design. Its insights are also applicable to the design of cost functions and performance metrics mo…
The EMPATHIC Framework for Task Learning from Implicit Human Feedback
Yuchen Cui, Qiping Zhang, Alessandro Allievi +3
Reactions such as gestures, facial expressions, and vocalizations are an abundant, naturally occurring channel of information that humans provide during interactions. A robot or ot…