Physics-informed attention-based neural network for solving non-linear partial differential equations
arXiv:2105.07898
Abstract
Physics-Informed Neural Networks (PINNs) have enabled significant improvements in modelling physical processes described by partial differential equations (PDEs). PINNs are based on simple architectures, and learn the behavior of complex physical systems by optimizing the network parameters to minimize the residual of the underlying PDE. Current network architectures share some of the limitations of classical numerical discretization schemes when applied to non-linear differential equations in continuum mechanics. A paradigmatic example is the solution of hyperbolic conservation laws that develop highly localized nonlinear shock waves. Learning solutions of PDEs with dominant hyperbolic character is a challenge for current PINN approaches, which rely, like most grid-based numerical schemes, on adding artificial dissipation. Here, we address the fundamental question of which network architectures are best suited to learn the complex behavior of non-linear PDEs. We focus on network architecture rather than on residual regularization. Our new methodology, called Physics-Informed Attention-based Neural Networks, (PIANNs), is a combination of recurrent neural networks and attention mechanisms. The attention mechanism adapts the behavior of the deep neural network to the non-linear features of the solution, and break the current limitations of PINNs. We find that PIANNs effectively capture the shock front in a hyperbolic model problem, and are capable of providing high-quality solutions inside and beyond the training set.
References in corpus (13)
- Adam: A Method for Stochastic Optimization
- Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation
- An Energy Approach to the Solution of Partial Differential Equations in Computational Mechanics via Machine Learning: Concepts, Implementation and Applications
- Solving high-dimensional partial differential equations using deep learning
- NSFnets (Navier-Stokes Flow nets): Physics-informed neural networks for the incompressible Navier-Stokes equations
- Physics Informed Deep Learning (Part I): Data-driven Solutions of Nonlinear Partial Differential Equations
- Learning data driven discretizations for partial differential equations
- Quantifying total uncertainty in physics-informed neural networks for solving forward and inverse stochastic problems
- Physics-informed Neural Networks for Solving Nonlinear Diffusivity and Biot's equations
- An overview on deep learning-based approximation methods for partial differential equations
- Physics Informed Deep Learning for Transport in Porous Media. Buckley Leverett Problem
- Conditional Karhunen-Loève expansion for uncertainty quantification and active learning in partial differential equation models
- Physics Informed Deep Learning for Flow and Transport in Porous Media