Physical Symmetries Embedded in Neural Networks

Abstract

Neural networks are a central technique in machine learning. Recent years have seen a wave of interest in applying neural networks to physical systems for which the governing dynamics are known and expressed through differential equations. Two fundamental challenges facing the development of neural networks in physics applications is their lack of interpretability and their physics-agnostic design. The focus of the present work is to embed physical constraints into the structure of the neural network to address the second fundamental challenge. By constraining tunable parameters (such as weights and biases) and adding special layers to the network, the desired constraints are guaranteed to be satisfied without the need for explicit regularization terms. This is demonstrated on upervised and unsupervised networks for two basic symmetries: even/odd symmetry of a function and energy conservation. In the supervised case, the network with embedded constraints is shown to perform well on regression problems while simultaneously obeying the desired constraints whereas a traditional network fits the data but violates the underlying constraints. Finally, a new unsupervised neural network is proposed that guarantees energy conservation through an embedded symplectic structure. The symplectic neural network is used to solve a system of energy-conserving differential equations and out-performs an unsupervised, non-symplectic neural network.

arXiv commentThis is the same manuscript with version 1 (arXiv:1904.08991v1) which accidentally was replaced 16 pages, 8 figures

References in corpus (8)

• DGM: A deep learning algorithm for solving partial differential equations
• Solving the Quantum Many-Body Problem with Artificial Neural Networks
• Machine Learning of Linear Differential Equations using Gaussian Processes
• Inferring solutions of differential equations using noisy multi-fidelity data
• Machine Learning Topological Invariants with Neural Networks
• Neural Networks Trained to Solve Differential Equations Learn General Representations
• Machine learning with observers predicts complex spatiotemporal behavior
• Deep learning for turbulent channel flow

Cited by in corpus (16)

• PPINN: Parareal Physics-Informed Neural Network for time-dependent PDEs
• AI Poincaré: Machine Learning Conservation Laws from Trajectories
• Incorporating Symmetry into Deep Dynamics Models for Improved Generalization
• Reservoir computing model of two-dimensional turbulent convection
• A neural network multigrid solver for the Navier-Stokes equations
• Physics-Driven Learning for Inverse Problems in Quantum Chromodynamics
• A probabilistic generative model for semi-supervised training of coarse-grained surrogates and enforcing physical constraints through virtual observables
• Neural Canonical Transformation with Symplectic Flows
• Classification of Periodic Variable Stars with Novel Cyclic-Permutation Invariant Neural Networks
• Structure Preservation for the Deep Neural Network Multigrid Solver
• Knowledge-data fusion dominated vehicle platoon dynamics modeling and analysis: A physics-encoded deep learning approach
• Joint Parameter Discovery and Generative Modeling of Dynamic Systems
• Beyond permutation equivariance in graph networks
• Unsupervised Learning of Solutions to Differential Equations with Generative Adversarial Networks
• Physical Constraint Embedded Neural Networks for inference and noise regulation
• Symmetry-driven graph neural networks