Learning Visual Predictive Models of Physics for Playing Billiards

Abstract

The ability to plan and execute goal specific actions in varied, unexpected settings is a central requirement of intelligent agents. In this paper, we explore how an agent can be equipped with an internal model of the dynamics of the external world, and how it can use this model to plan novel actions by running multiple internal simulations ("visual imagination"). Our models directly process raw visual input, and use a novel object-centric prediction formulation based on visual glimpses centered on objects (fixations) to enforce translational invariance of the learned physical laws. The agent gathers training data through random interaction with a collection of different environments, and the resulting model can then be used to plan goal-directed actions in novel environments that the agent has not seen before. We demonstrate that our agent can accurately plan actions for playing a simulated billiards game, which requires pushing a ball into a target position or into collision with another ball.

References in corpus (3)

• Continuous control with deep reinforcement learning
• End-to-End Training of Deep Visuomotor Policies
• Action-Conditional Video Prediction using Deep Networks in Atari Games

Cited by in corpus (28)

• From Perception to Decision: A Data-driven Approach to End-to-end Motion Planning for Autonomous Ground Robots
• A Review on Deep Learning Techniques for Video Prediction
• Graph networks as learnable physics engines for inference and control
• Learning to Predict the Cosmological Structure Formation
• Flexible Neural Representation for Physics Prediction
• Relational Neural Expectation Maximization: Unsupervised Discovery of Objects and their Interactions
• ThreeDWorld: A Platform for Interactive Multi-Modal Physical Simulation
• Learning model-based planning from scratch
• Visual Interaction Networks
• To Fall Or Not To Fall: A Visual Approach to Physical Stability Prediction
• Blending Diverse Physical Priors with Neural Networks
• Human-Level Reinforcement Learning through Theory-Based Modeling, Exploration, and Planning
• Prediction and Control with Temporal Segment Models
• Structured Object-Aware Physics Prediction for Video Modeling and Planning
• Motion Prediction Under Multimodality with Conditional Stochastic Networks
• Learning to Represent Mechanics via Long-term Extrapolation and Interpolation
• Visual Stability Prediction and Its Application to Manipulation
• CausalCity: Complex Simulations with Agency for Causal Discovery and Reasoning
• Learning Long-term Visual Dynamics with Region Proposal Interaction Networks
• Acquiring Target Stacking Skills by Goal-Parameterized Deep Reinforcement Learning
• ShapeStacks: Learning Vision-Based Physical Intuition for Generalised Object Stacking
• Block Contextual MDPs for Continual Learning
• CLOUD: Contrastive Learning of Unsupervised Dynamics
• Interpretable Intuitive Physics Model
• Sound2Sight: Generating Visual Dynamics from Sound and Context
• Spatio-Temporal Image Boundary Extrapolation
• A New Framework for Machine Intelligence: Concepts and Prototype
• Computational principles of intelligence: learning and reasoning with neural networks