Reward Tampering Problems and Solutions in Reinforcement Learning: A Causal Influence Diagram Perspective

Tom Everitt, Marcus Hutter, Ramana Kumar +1

Can humans get arbitrarily capable reinforcement learning (RL) agents to do their bidding? Or will sufficiently capable RL agents always find ways to bypass their intended objectiv…