Can Transfer Neuroevolution Tractably Solve Your Differential Equations?

Abstract

This paper introduces neuroevolution for solving differential equations. The solution is obtained through optimizing a deep neural network whose loss function is defined by the residual terms from the differential equations. Recent studies have focused on learning such physics-informed neural networks through stochastic gradient descent (SGD) variants, yet they face the difficulty of obtaining an accurate solution due to optimization challenges. In the context of solving differential equations, we are faced with the problem of finding globally optimum parameters of the network, instead of being concerned with out-of-sample generalization. SGD, which searches along a single gradient direction, is prone to become trapped in local optima, so it may not be the best approach here. In contrast, neuroevolution carries out a parallel exploration of diverse solutions with the goal of circumventing local optima. It could potentially find more accurate solutions with better optimized neural networks. However, neuroevolution can be slow, raising tractability issues in practice. With that in mind, a novel and computationally efficient transfer neuroevolution algorithm is proposed in this paper. Our method is capable of exploiting relevant experiential priors when solving a new problem, with adaptation to protect against the risk of negative transfer. The algorithm is applied on a variety of differential equations to empirically demonstrate that transfer neuroevolution can indeed achieve better accuracy and faster convergence than SGD. The experimental outcomes thus establish transfer neuroevolution as a noteworthy approach for solving differential equations, one that has never been studied in the past. Our work expands the resource of available algorithms for optimizing physics-informed neural networks.

References in corpus (20)

• Adam: A Method for Stochastic Optimization
• Artificial Neural Networks for Solving Ordinary and Partial Differential Equations
• DGM: A deep learning algorithm for solving partial differential equations
• Adaptive activation functions accelerate convergence in deep and physics-informed neural networks
• Evolution Strategies as a Scalable Alternative to Reinforcement Learning
• Physics-informed neural networks for inverse problems in nano-optics and metamaterials
• A unified deep artificial neural network approach to partial differential equations in complex geometries
• Machine learning in cardiovascular flows modeling: Predicting arterial blood pressure from non-invasive 4D flow MRI data using physics-informed neural networks
• PathNet: Evolution Channels Gradient Descent in Super Neural Networks
• Adversarial Uncertainty Quantification in Physics-Informed Neural Networks
• Deep Hidden Physics Models: Deep Learning of Nonlinear Partial Differential Equations
• Deep Learning of Vortex Induced Vibrations
• Modeling the Dynamics of PDE Systems with Physics-Constrained Deep Auto-Regressive Networks
• Artificial Neural Network Methods in Quantum Mechanics
• D3M: A deep domain decomposition method for partial differential equations
• Scaling MAP-Elites to Deep Neuroevolution
• On the Relationship Between the OpenAI Evolution Strategy and Stochastic Gradient Descent
• Robust Optimization through Neuroevolution
• Machine Discovery of Partial Differential Equations from Spatiotemporal Data
• AIR5: Five Pillars of Artificial Intelligence Research

Cited by in corpus (7)

• CAN-PINN: A Fast Physics-Informed Neural Network Based on Coupled-Automatic-Numerical Differentiation Method
• Learning in Sinusoidal Spaces with Physics-Informed Neural Networks
• Scalable Transfer Evolutionary Optimization: Coping with Big Task Instances
• SVD-PINNs: Transfer Learning of Physics-Informed Neural Networks via Singular Value Decomposition
• Evolutionary Optimization of Physics-Informed Neural Networks: Evo-PINN Frontiers and Opportunities
• Neuroevolution of Physics-Informed Neural Nets: Benchmark Problems and Comparative Results
• Deep learning in physics: a study of dielectric quasi-cubic particles in a uniform electric field