Unsupervised Learning of Lagrangian Dynamics from Images for Prediction and Control
arXiv:2007.01926
Abstract
Recent approaches for modelling dynamics of physical systems with neural networks enforce Lagrangian or Hamiltonian structure to improve prediction and generalization. However, when coordinates are embedded in high-dimensional data such as images, these approaches either lose interpretability or can only be applied to one particular example. We introduce a new unsupervised neural network model that learns Lagrangian dynamics from images, with interpretability that benefits prediction and control. The model infers Lagrangian dynamics on generalized coordinates that are simultaneously learned with a coordinate-aware variational autoencoder (VAE). The VAE is designed to account for the geometry of physical systems composed of multiple rigid bodies in the plane. By inferring interpretable Lagrangian dynamics, the model learns physical system properties, such as kinetic and potential energy, which enables long-term prediction of dynamics in the image space and synthesis of energy-based controllers.
This version corrects an error in Equation (3) of the 2020 NeurIPS Proceedings paper
References in corpus (9)
- PyTorch: An Imperative Style, High-Performance Deep Learning Library
- Interaction Networks for Learning about Objects, Relations and Physics
- Symplectic ODE-Net: Learning Hamiltonian Dynamics with Control
- A Disentangled Recognition and Nonlinear Dynamics Model for Unsupervised Learning
- Unsupervised Learning of Object Keypoints for Perception and Control
- Quantifying Translation-Invariance in Convolutional Neural Networks
- Hamiltonian Graph Networks with ODE Integrators
- Dissipative SymODEN: Encoding Hamiltonian Dynamics with Dissipation and Control into Deep Learning
- Variational Integrator Networks for Physically Structured Embeddings