Visual Reaction: Learning to Play Catch with Your Drone
arXiv:1912.02155
Abstract
In this paper we address the problem of visual reaction: the task of interacting with dynamic environments where the changes in the environment are not necessarily caused by the agent itself. Visual reaction entails predicting the future changes in a visual environment and planning accordingly. We study the problem of visual reaction in the context of playing catch with a drone in visually rich synthetic environments. This is a challenging problem since the agent is required to learn (1) how objects with different physical properties and shapes move, (2) what sequence of actions should be taken according to the prediction, (3) how to adjust the actions based on the visual feedback from the dynamic environment (e.g., when objects bouncing off a wall), and (4) how to reason and act with an unexpected state change in a timely manner. We propose a new dataset for this task, which includes 30K throws of 20 types of objects in different directions with different forces. Our results show that our model that integrates a forecaster with a planner outperforms a set of strong baselines that are based on tracking as well as pure model-based and model-free RL baselines. The code and dataset are available at github.com/KuoHaoZeng/Visual_Reaction.
arXiv comment8 pages, 6 figures
References in corpus (13)
- Adam: A Method for Stochastic Optimization
- Unsupervised Learning of Video Representations using LSTMs
- Decomposing Motion and Content for Natural Video Sequence Prediction
- Learning to Navigate in Complex Environments
- Learning Continuous Control Policies by Stochastic Value Gradients
- Learning Latent Dynamics for Planning from Pixels
- Continuous Deep Q-Learning with Model-based Acceleration
- Model-Ensemble Trust-Region Policy Optimization
- The Predictron: End-To-End Learning and Planning
- Visual Dynamics: Probabilistic Future Frame Synthesis via Cross Convolutional Networks
- Combining Model-Based and Model-Free Updates for Trajectory-Centric Reinforcement Learning
- Learning to Act by Predicting the Future
- Learning and Querying Fast Generative Models for Reinforcement Learning