Iterative Surrogate Model Optimization (ISMO): An active learning algorithm for PDE constrained optimization with deep neural networks

Abstract

We present a novel active learning algorithm, termed as iterative surrogate model optimization (ISMO), for robust and efficient numerical approximation of PDE constrained optimization problems. This algorithm is based on deep neural networks and its key feature is the iterative selection of training data through a feedback loop between deep neural networks and any underlying standard optimization algorithm. Under suitable hypotheses, we show that the resulting optimizers converge exponentially fast (and with exponentially decaying variance), with respect to increasing number of training samples. Numerical examples for optimal control, parameter identification and shape optimization problems for PDEs are provided to validate the proposed theory and to illustrate that ISMO significantly outperforms a standard deep neural network based surrogate optimization algorithm.

References in corpus (9)

• SciPy 1.0--Fundamental Algorithms for Scientific Computing in Python
• Artificial Neural Networks for Solving Ordinary and Partial Differential Equations
• Solving high-dimensional partial differential equations using deep learning
• Hidden Physics Models: Machine Learning of Nonlinear Partial Differential Equations
• Deep learning-based numerical methods for high-dimensional parabolic partial differential equations and backward stochastic differential equations
• Accelerating Eulerian Fluid Simulation With Convolutional Networks
• Deep learning observables in computational fluid dynamics
• Hidden Fluid Mechanics: A Navier-Stokes Informed Deep Learning Framework for Assimilating Flow Visualization Data
• A machine learning framework for data driven acceleration of computations of differential equations

Cited by in corpus (15)

• Physics Informed Neural Networks for Simulating Radiative Transfer
• An overview on deep learning-based approximation methods for partial differential equations
• On universal approximation and error bounds for Fourier Neural Operators
• Estimates on the generalization error of Physics Informed Neural Networks (PINNs) for approximating PDEs
• Instantaneous physics-based ground motion maps using reduced-order modeling
• Failure-averse Active Learning for Physics-constrained Systems
• Analysis of Heterogeneous Structures of Non-separated Scales on Curved Bridge Nodes
• Modelling parametric uncertainty in PDEs models via Physics-Informed Neural Networks
• A globally convergent method to accelerate large-scale optimization using on-the-fly model hyperreduction: application to shape optimization
• LSA-PINN: Linear Boundary Connectivity Loss for Solving PDEs on Complex Geometry
• An operator learning perspective on parameter-to-observable maps
• Higher-order Quasi-Monte Carlo Training of Deep Neural Networks
• Physics Informed Neural Networks (PINNs)for approximating nonlinear dispersive PDEs
• On the approximation of functions by tanh neural networks
• Estimates on the generalization error of Physics Informed Neural Networks (PINNs) for approximating a class of inverse problems for PDEs