Enforcing Analytic Constraints in Neural-Networks Emulating Physical Systems

Abstract

Neural networks can emulate nonlinear physical systems with high accuracy, yet they may produce physically-inconsistent results when violating fundamental constraints. Here, we introduce a systematic way of enforcing nonlinear analytic constraints in neural networks via constraints in the architecture or the loss function. Applied to convective processes for climate modeling, architectural constraints enforce conservation laws to within machine precision without degrading performance. Enforcing constraints also reduces errors in the subsets of the outputs most impacted by the constraints.

21 pages, 11 figures, 9 tables. Submitted to Physical Review Letters

References in corpus (14)

• Adam: A Method for Stochastic Optimization
• TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
• Theory-guided Data Science: A New Paradigm for Scientific Discovery from Data
• Deep learning to represent sub-grid processes in climate models
• Surrogate Modeling for Fluid Flows Based on Physics-Constrained Deep Learning Without Simulation Data
• Physics Informed Deep Learning (Part I): Data-driven Solutions of Nonlinear Partial Differential Equations
• Learning data driven discretizations for partial differential equations
• A Comprehensive Physics-Informed Machine Learning Framework for Predictive Turbulence Modeling
• Physics-Informed Machine Learning Approach for Augmenting Turbulence Models: A Comprehensive Framework
• Stable machine-learning parameterization of subgrid processes for climate modeling at a range of resolutions
• Physics-guided Neural Networks (PGNN): An Application in Lake Temperature Modeling
• Enforcing Statistical Constraints in Generative Adversarial Networks for Modeling Chaotic Dynamical Systems
• Achieving Conservation of Energy in Neural Network Emulators for Climate Modeling
• Coupled online learning as a way to tackle instabilities and biases in neural network parameterizations

Cited by in corpus (44)

• Bridging observation, theory and numerical simulation of the ocean using Machine Learning
• Theory-guided hard constraint projection (HCP): a knowledge-based data-driven scientific machine learning method
• Data-driven recovery of hidden physics in reduced order modeling of fluid flows
• On Recurrent Neural Networks for learning-based control: recent results and ideas for future developments
• Incorporating Symmetry into Deep Dynamics Models for Improved Generalization
• SPT-NRTL: A physics-guided machine learning model to predict thermodynamically consistent activity coefficients
• Model calibration using ESEm v1.0.0 -- an open, scalable Earth System Emulator
• Physics-informed machine learning for Structural Health Monitoring
• Learning physics-constrained subgrid-scale closures in the small-data regime for stable and accurate LES
• Machine learning for weather and climate are worlds apart
• Machine-learning hidden symmetries
• Explaining the physics of transfer learning a data-driven subgrid-scale closure to a different turbulent flow
• Accelerating hydrodynamic simulations of urban drainage systems with physics-guided machine learning
• Super-resolution data assimilation
• Towards physically consistent data-driven weather forecasting: Integrating data assimilation with equivariance-preserving deep spatial transformers
• Symplectic Learning for Hamiltonian Neural Networks
• Differentiable Programming for Earth System Modeling
• Machine learning applications for weather and climate need greater focus on extremes
• Improving deep learning with prior knowledge and cognitive models: A survey on enhancing explainability, adversarial robustness and zero-shot learning
• Equation-free surrogate modeling of geophysical flows at the intersection of machine learning and data assimilation
• Learning Closed-form Equations for Subgrid-scale Closures from High-fidelity Data: Promises and Challenges
• Learning Physical Models that Can Respect Conservation Laws
• Physics-consistent machine learning: output projection onto physical manifolds
• Convolutional LSTM Neural Networks for Modeling Wildland Fire Dynamics
• Invariant deep neural networks under the finite group for solving partial differential equations
• Training neural networks under physical constraints using a stochastic augmented Lagrangian approach
• Exact conservation laws for neural network integrators of dynamical systems
• Can we integrate spatial verification methods into neural-network loss functions for atmospheric science?
• Error-correcting neural networks for semi-Lagrangian advection in the level-set method
• STAN: Spatio-Temporal Attention Network for Pandemic Prediction Using Real World Evidence
• CIRA Guide to Custom Loss Functions for Neural Networks in Environmental Sciences -- Version 1
• Augmenting a Physics-Informed Neural Network for the 2D Burgers Equation by Addition of Solution Data Points
• PCE-PINNs: Physics-Informed Neural Networks for Uncertainty Propagation in Ocean Modeling
• Domain-informed neural networks for interaction localization within astroparticle experiments
• Evaluation, Tuning and Interpretation of Neural Networks for Meteorological Applications
• Label-free timing analysis of SiPM-based modularized detectors with physics-constrained deep learning
• MC-LSTM: Mass-Conserving LSTM
• A Mechanism-Learning Deeply Coupled Model for Remote Sensing Retrieval of Global Land Surface Temperature
• A Physics-Constrained Neural Differential Equation Framework for Data-Driven Snowpack Simulation
• Electron neural closure for turbulent magnetosheath simulations: energy channels
• A kinetic-based regularization method for data science applications
• A Data-Driven Statistical Description for the Hydrodynamics of Active Matter
• Real-time reconstruction of intense, ultrafast laser pulses using deep learning
• Error-Correcting Neural Networks for Two-Dimensional Curvature Computation in the Level-Set Method