Incorporating Symmetry into Deep Dynamics Models for Improved Generalization

Abstract

Recent work has shown deep learning can accelerate the prediction of physical dynamics relative to numerical solvers. However, limited physical accuracy and an inability to generalize under distributional shift limit its applicability to the real world. We propose to improve accuracy and generalization by incorporating symmetries into convolutional neural networks. Specifically, we employ a variety of methods each tailored to enforce a different symmetry. Our models are both theoretically and experimentally robust to distributional shift by symmetry group transformations and enjoy favorable sample complexity. We demonstrate the advantage of our approach on a variety of physical dynamics including Rayleigh Bénard convection and real-world ocean currents and temperatures. Compared with image or text applications, our work is a significant step towards applying equivariant neural networks to high-dimensional systems with complex dynamics. We open-source our simulation, data, and code at \url{https://github.com/Rose-STL-Lab/Equivariant-Net}.

References in corpus (25)

• Deep Residual Learning for Image Recognition
• Physics Informed Deep Learning (Part I): Data-driven Solutions of Nonlinear Partial Differential Equations
• Group Equivariant Convolutional Networks
• Enforcing Analytic Constraints in Neural-Networks Emulating Physical Systems
• Decomposing Motion and Content for Natural Video Sequence Prediction
• A Review on Deep Learning Techniques for Video Prediction
• Accelerating Eulerian Fluid Simulation With Convolutional Networks
• On the Generalization of Equivariance and Convolution in Neural Networks to the Action of Compact Groups
• Enforcing Statistical Constraints in Generative Adversarial Networks for Modeling Chaotic Dynamical Systems
• Exploiting Cyclic Symmetry in Convolutional Neural Networks
• Visual Dynamics: Probabilistic Future Frame Synthesis via Cross Convolutional Networks
• Deep unsupervised learning of turbulence for inflow generation at various Reynolds numbers
• Deep Dynamical Modeling and Control of Unsteady Fluid Flows
• Invariant Representations without Adversarial Training
• General $E(2)$-Equivariant Steerable CNNs
• A Kernel Theory of Modern Data Augmentation
• Generalizing Convolutional Neural Networks for Equivariance to Lie Groups on Arbitrary Continuous Data
• Compressed Convolutional LSTM: An Efficient Deep Learning framework to Model High Fidelity 3D Turbulence
• Physical Symmetries Embedded in Neural Networks
• Deep Scale-spaces: Equivariance Over Scale
• Scale Steerable Filters for Locally Scale-Invariant Convolutional Neural Networks
• Rotation Equivariance and Invariance in Convolutional Neural Networks
• Deep learning for turbulent channel flow
• Predictive State Recurrent Neural Networks
• Equivariant neural networks and equivarification

Cited by in corpus (18)

• Learning physics-constrained subgrid-scale closures in the small-data regime for stable and accurate LES
• Machine Learning for Physics-Informed Generation of Dispersed Multiphase Flow Using Generative Adversarial Networks
• Physics-constrained Deep Learning for Robust Inverse ECG Modeling
• Bridging Physics-based and Data-driven modeling for Learning Dynamical Systems
• Bounded nonlinear forecasts of partially observed geophysical systems with physics-constrained deep learning
• Machine learning and invariant theory
• Coordinate Independent Convolutional Networks -- Isometry and Gauge Equivariant Convolutions on Riemannian Manifolds
• Trajectory Prediction using Equivariant Continuous Convolution
• Frame invariance and scalability of neural operators for partial differential equations
• Meta-Learning Dynamics Forecasting Using Task Inference
• Neural Pharmacodynamic State Space Modeling
• Learning dissection trajectories from expert surgical videos via imitation learning with equivariant diffusion
• Equivariant Learning of Stochastic Fields: Gaussian Processes and Steerable Conditional Neural Processes
• Symmetry constrained neural networks for detection and localization of damage in metal plates
• Kinematically consistent recurrent neural networks for learning inverse problems in wave propagation
• A Use of Even Activation Functions in Neural Networks
• Rotational and Reflectional Equivariant Convolutional Neural Network for data-limited applications: Multiphase Flow demonstration
• Learning Similarity Metrics for Volumetric Simulations with Multiscale CNNs