Training an Interactive Helper

Abstract

Developing agents that can quickly adapt their behavior to new tasks remains a challenge. Meta-learning has been applied to this problem, but previous methods require either specifying a reward function which can be tedious or providing demonstrations which can be inefficient. In this paper, we investigate if, and how, a "helper" agent can be trained to interactively adapt their behavior to maximize the reward of another agent, whom we call the "prime" agent, without observing their reward or receiving explicit demonstrations. To this end, we propose to meta-learn a helper agent along with a prime agent, who, during training, observes the reward function and serves as a surrogate for a human prime. We introduce a distribution of multi-agent cooperative foraging tasks, in which only the prime agent knows the objects that should be collected. We demonstrate that, from the emerged physical communication, the trained helper rapidly infers and collects the correct objects.

The paper "Learning to Interactively Learn and Assist" (LILA), at arXiv:1906.10187, supersedes this paper. This preliminary workshop paper appeared in the Emergent Communication Workshop and Workshop on Learning by Instruction at NeurIPS 2018

References in corpus (7)

• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
• Playing Atari with Deep Reinforcement Learning
• QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulation
• Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Review
• Continuous Deep Q-Learning with Model-based Acceleration
• Reinforcement Learning with Unsupervised Auxiliary Tasks
• Task-Embedded Control Networks for Few-Shot Imitation Learning