Uncovering Closed-form Governing Equations of Nonlinear Dynamics from Videos

Abstract

Distilling analytical models from data has the potential to advance our understanding and prediction of nonlinear dynamics. Although discovery of governing equations based on observed system states (e.g., trajectory time series) has revealed success in a wide range of nonlinear dynamics, uncovering the closed-form equations directly from raw videos still remains an open challenge. To this end, we introduce a novel end-to-end unsupervised deep learning framework to uncover the mathematical structure of equations that governs the dynamics of moving objects in videos. Such an architecture consists of (1) an encoder-decoder network that learns low-dimensional spatial/pixel coordinates of the moving object, (2) a learnable Spatial-Physical Transformation component that creates mapping between the extracted spatial/pixel coordinates and the latent physical states of dynamics, and (3) a numerical integrator-based sparse regression module that uncovers the parsimonious closed-form governing equations of learned physical states and, meanwhile, serves as a constraint to the autoencoder. The efficacy of the proposed method is demonstrated by uncovering the governing equations of a variety of nonlinear dynamical systems depicted by moving objects in videos. The resulting computational framework enables discovery of parsimonious interpretable model in a flexible and accessible sensing environment where only videos are available.

arXiv comment25 pages

References in corpus (19)

• Discovering governing equations from data: Sparse identification of nonlinear dynamical systems
• Physics-informed learning of governing equations from scarce data
• PDE-Net 2.0: Learning PDEs from Data with A Numeric-Symbolic Hybrid Deep Network
• SINDy-PI: A Robust Algorithm for Parallel Implicit Sparse Identification of Nonlinear Dynamics
• Deep learning of dynamics and signal-noise decomposition with time-stepping constraints
• Discovery of Nonlinear Multiscale Systems: Sampling Strategies and Embeddings
• A Compositional Object-Based Approach to Learning Physical Dynamics
• Relational Neural Expectation Maximization: Unsupervised Discovery of Objects and their Interactions
• Sequential Attend, Infer, Repeat: Generative Modelling of Moving Objects
• A Data-Driven Approach for Discovering Stochastic Dynamical Systems with Non-Gaussian Levy Noise
• Symbolic Pregression: Discovering Physical Laws from Distorted Video
• Unsupervised Learning of Latent Physical Properties Using Perception-Prediction Networks
• Learning A Physical Long-term Predictor
• PVEs: Position-Velocity Encoders for Unsupervised Learning of Structured State Representations
• Sparse Identification of Truncation Errors
• Data-driven discovery of partial differential equation models with latent variables
• Object-Oriented Dynamics Predictor
• Physics-as-Inverse-Graphics: Unsupervised Physical Parameter Estimation from Video
• Grounding Physical Concepts of Objects and Events Through Dynamic Visual Reasoning