Improved Training of Physics-Informed Neural Networks with Model Ensembles

Abstract

Learning the solution of partial differential equations (PDEs) with a neural network is an attractive alternative to traditional solvers due to its elegance, greater flexibility and the ease of incorporating observed data. However, training such physics-informed neural networks (PINNs) is notoriously difficult in practice since PINNs often converge to wrong solutions. In this paper, we address this problem by training an ensemble of PINNs. Our approach is motivated by the observation that individual PINN models find similar solutions in the vicinity of points with targets (e.g., observed data or initial conditions) while their solutions may substantially differ farther away from such points. Therefore, we propose to use the ensemble agreement as the criterion for gradual expansion of the solution interval, that is including new points for computing the loss derived from differential equations. Due to the flexibility of the domain expansion, our algorithm can easily incorporate measurements in arbitrary locations. In contrast to the existing PINN algorithms with time-adaptive strategies, the proposed algorithm does not need a pre-defined schedule of interval expansion and it treats time and space equally. We experimentally show that the proposed algorithm can stabilize PINN training and yield performance competitive to the recent variants of PINNs trained with time adaptation.

References in corpus (9)

• Improving neural networks by preventing co-adaptation of feature detectors
• Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains
• On the eigenvector bias of Fourier feature networks: From regression to solving multi-scale PDEs with physics-informed neural networks
• A comprehensive study of non-adaptive and residual-based adaptive sampling for physics-informed neural networks
• Learning with Pseudo-Ensembles
• A Physics Informed Neural Network for Time-Dependent Nonlinear and Higher Order Partial Differential Equations
• A Method for Representing Periodic Functions and Enforcing Exactly Periodic Boundary Conditions with Deep Neural Networks
• Self-Adaptive Physics-Informed Neural Networks using a Soft Attention Mechanism
• Learning in Sinusoidal Spaces with Physics-Informed Neural Networks

Cited by in corpus (3)

• A comprehensive study of non-adaptive and residual-based adaptive sampling for physics-informed neural networks
• Lagrangian PINNs: A causality-conforming solution to failure modes of physics-informed neural networks
• Stability in Training PINNs for Stiff PDEs: Why Initial Conditions Matter