Fredholm Neural Networks

Abstract

Within the family of explainable machine-learning, we present Fredholm neural networks (Fredholm NNs): deep neural networks (DNNs) architectures motivated by fixed-point iteration schemes for the solution of linear and nonlinear Fredholm integral equations (FIEs) of the second kind. We also show how the proposed framework can be used for the solution of inverse problems. Applications of FIEs include the solution of ordinary, as well as partial differential equations (ODEs, PDEs) and many more. We first prove that Fredholm NNs provide accurate solutions. We then provide insight into the values of the hyperparameters and trainable/explainable weights and biases of the DNN, by directly connecting their values to the underlying mathematical theory. For our illustrations, we use Fredholm NNs to solve both linear and nonlinear problems, including elliptic PDEs and boundary value problems. We show that the proposed scheme achieves significant numerical approximation accuracy across both the domain and boundary. The proposed methodology provides insight into the connection between neural networks and classical numerical methods, and we posit that it can have applications in fields such as Uncertainty Quantification (UQ) and explainable artificial intelligence (XAI). Thus, we believe that it will trigger further advances in the intersection between scientific machine learning and numerical analysis.

The final version of this preprint with updated examples can be found in the journal publication

References in corpus (13)

• DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators
• Artificial Neural Networks for Solving Ordinary and Partial Differential Equations
• Solving high-dimensional partial differential equations using deep learning
• Uncertainty Quantification in Scientific Machine Learning: Methods, Metrics, and Comparisons
• Local Extreme Learning Machines and Domain Decomposition for Solving Linear and Nonlinear Partial Differential Equations
• Neural Network with Unbounded Activation Functions is Universal Approximator
• Stiff Neural Ordinary Differential Equations
• Neural-net-induced Gaussian process regression for function approximation and PDE solution
• Coarse-scale PDEs from fine-scale observations via machine learning
• Constructing coarse-scale bifurcation diagrams from spatio-temporal observations of microscopic simulations: A parsimonious machine learning approach
• A Modified Batch Intrinsic Plasticity Method for Pre-training the Random Coefficients of Extreme Learning Machines
• Neural Integral Equations
• RandONet: Shallow-Networks with Random Projections for learning linear and nonlinear operators