Learning to Request Guidance in Emergent Communication
arXiv:1912.05525
Abstract
Previous research into agent communication has shown that a pre-trained guide can speed up the learning process of an imitation learning agent. The guide achieves this by providing the agent with discrete messages in an emerged language about how to solve the task. We extend this one-directional communication by a one-bit communication channel from the learner back to the guide: It is able to ask the guide for help, and we limit the guidance by penalizing the learner for these requests. During training, the agent learns to control this gate based on its current observation. We find that the amount of requested guidance decreases over time and guidance is requested in situations of high uncertainty. We investigate the agent's performance in cases of open and closed gates and discuss potential motives for the observed gating behavior.
References in corpus (10)
- Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1
- Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation
- Emergence of Grounded Compositional Language in Multi-Agent Populations
- Scalable agent alignment via reward modeling: a research direction
- Listen, Attend, and Walk: Neural Mapping of Navigational Instructions to Action Sequences
- Interactive Grounded Language Acquisition and Generalization in a 2D World
- Active Reinforcement Learning: Observing Rewards at a Cost
- Active Reinforcement Learning with Monte-Carlo Tree Search
- Delegative Reinforcement Learning: learning to avoid traps with a little help
- Mastering emergent language: learning to guide in simulated navigation