RL agents Implicitly Learning Human Preferences
arXiv:2002.06137
Abstract
In the real world, RL agents should be rewarded for fulfilling human preferences. We show that RL agents implicitly learn the preferences of humans in their environment. Training a classifier to predict if a simulated human's preferences are fulfilled based on the activations of a RL agent's neural network gets .93 AUC. Training a classifier on the raw environment state gets only .8 AUC. Training the classifier off of the RL agent's activations also does much better than training off of activations from an autoencoder. The human preference classifier can be used as the reward function of an RL agent to make RL agent more beneficial for humans.
References in corpus (8)
- Solving Rubik's Cube with a Robot Hand
- Deep reinforcement learning from human preferences
- Deep Transfer Learning: A new deep learning glitch classification method for advanced LIGO
- Glitch Classification and Clustering for LIGO with Deep Transfer Learning
- Visualizing and Understanding Atari Agents
- Modeling Others using Oneself in Multi-Agent Reinforcement Learning
- Graying the black box: Understanding DQNs
- Agent Modeling as Auxiliary Task for Deep Reinforcement Learning