91 citations · 189 across the 19 of their papers we have counts for
4 papers · 1 filter
Optimal Policies Tend to Seek Power
Alexander Matt Turner, Logan Smith, Rohin Shah +2
Some researchers speculate that intelligent reinforcement learning (RL) agents would be incentivized to seek resources and power in pursuit of their objectives. Other researchers p…
On the Utility of Learning about Humans for Human-AI Coordination
Micah Carroll, Rohin Shah, Mark K. Ho +4
While we would like agents that can coordinate with humans, current algorithms such as self-play and population-based training create agents that can coordinate with themselves. Ag…
On the Feasibility of Learning, Rather than Assuming, Human Biases for Reward Inference
Rohin Shah, Noah Gundotra, Pieter Abbeel +1
Our goal is for agents to optimize the right reward function, despite how difficult it is for us to specify what that is. Inverse Reinforcement Learning (IRL) enables us to infer r…
Preferences Implicit in the State of the World
Rohin Shah, Dmitrii Krasheninnikov, Jordan Alexander +2
Reinforcement learning (RL) agents optimize only the features specified in a reward function and are indifferent to anything left out inadvertently. This means that we must not onl…