Sidekick Policy Learning for Active Visual Exploration
arXiv:1807.11010
Abstract
We consider an active visual exploration scenario, where an agent must intelligently select its camera motions to efficiently reconstruct the full environment from only a limited set of narrow field-of-view glimpses. While the agent has full observability of the environment during training, it has only partial observability once deployed, being constrained by what portions it has seen and what camera motions are permissible. We introduce sidekick policy learning to capitalize on this imbalance of observability. The main idea is a preparatory learning phase that attempts simplified versions of the eventual exploration task, then guides the agent via reward shaping or initial policy supervision. To support interpretation of the resulting policies, we also develop a novel policy visualization technique. Results on active visual exploration tasks with 360 scenes and 3D objects show that sidekicks consistently improve performance and convergence rates over existing methods. Code, data and demos are available.
arXiv comment26 pages, 13 figures, to appear in ECCV 2018
References in corpus (13)
- Distilling the Knowledge in a Neural Network
- Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
- UCF101: A Dataset of 101 Human Actions Classes From Videos in The Wild
- End to End Learning for Self-Driving Cars
- Recurrent Models of Visual Attention
- Multiple Object Recognition with Visual Attention
- CARLA: An Open Urban Driving Simulator
- Unifying Count-Based Exploration and Intrinsic Motivation
- Thinking Fast and Slow with Deep Learning and Tree Search
- Visualizing and Understanding Atari Agents
- Learning Using Privileged Information: SVM+ and Weighted SVM
- Unifying Map and Landmark Based Representations for Visual Navigation
- Visual Semantic Planning using Deep Successor Representations