95 citations · 124 across the 4 of their papers we have counts for
4 papers
Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedback
Stephen Casper, Xander Davies, Claudia Shi +29
Reinforcement learning from human feedback (RLHF) is a technique for training AI systems to align with human goals. RLHF has emerged as the central method used to finetune state-of…
Goal Misgeneralization in Deep Reinforcement Learning
Lauro Langosco, Jack Koch, Lee Sharkey +3
We study goal misgeneralization, a type of out-of-distribution generalization failure in reinforcement learning (RL). Goal misgeneralization failures occur when an RL agent retains…
Robust Semantic Interpretability: Revisiting Concept Activation Vectors
Jacob Pfau, Albert T. Young, Jerome Wei +2
Interpretability methods for image classification assess model trustworthiness by attempting to expose whether the model is systematically biased or attending to the same cues as a…
Global Saliency: Aggregating Saliency Maps to Assess Dataset Artefact Bias
Jacob Pfau, Albert T. Young, Maria L. Wei +1
In high-stakes applications of machine learning models, interpretability methods provide guarantees that models are right for the right reasons. In medical imaging, saliency maps h…