Interactive Learning from Activity Description
arXiv:2102.07024
Abstract
We present a novel interactive learning protocol that enables training request-fulfilling agents by verbally describing their activities. Unlike imitation learning (IL), our protocol allows the teaching agent to provide feedback in a language that is most appropriate for them. Compared with reward in reinforcement learning (RL), the description feedback is richer and allows for improved sample complexity. We develop a probabilistic framework and an algorithm that practically implements our protocol. Empirical results in two challenging request-fulfilling problems demonstrate the strengths of our approach: compared with RL baselines, it is more sample-efficient; compared with IL baselines, it achieves competitive success rates without requiring the teaching agent to be able to demonstrate the desired behavior using the learning agent's actions. Apart from empirical evaluation, we also provide theoretical guarantees for our algorithm under certain assumptions about the teacher and the environment.
arXiv commentICML 2021
References in corpus (20)
- Theoretical Models of Learning to Learn
- Learning to Communicate with Deep Multi-Agent Reinforcement Learning
- Hindsight Experience Replay
- Grounded Language Learning in a Simulated 3D World
- Third-Person Imitation Learning
- Listen, Attend, and Walk: Neural Mapping of Navigational Instructions to Action Sequences
- Language as an Abstraction for Hierarchical Deep Reinforcement Learning
- Gated-Attention Architectures for Task-Oriented Language Grounding
- BabyAI: A Platform to Study the Sample Efficiency of Grounded Language Learning
- From Language to Goals: Inverse Reinforcement Learning for Vision-Based Instruction Following
- Language as a Cognitive Tool to Imagine Goals in Curiosity-Driven Exploration
- Reinforcement Learning with Convex Constraints
- Reasoning About Pragmatics with Neural Listeners and Speakers
- FLAMBE: Structural Complexity and Representation Learning of Low Rank MDPs
- Teaching Machines to Describe Images via Natural Language Feedback
- General Evaluation for Instruction Conditioned Navigation using Dynamic Time Warping
- ACTRCE: Augmenting Experience via Teacher's Advice For Multi-Goal Reinforcement Learning
- Provably Efficient Imitation Learning from Observation Alone
- Interactive Task and Concept Learning from Natural Language Instructions and GUI Demonstrations
- Inverse Reinforcement Learning with Natural Language Goals