Bayesian Deep Convolutional Encoder-Decoder Networks for Surrogate Modeling and Uncertainty Quantification

Abstract

We are interested in the development of surrogate models for uncertainty quantification and propagation in problems governed by stochastic PDEs using a deep convolutional encoder-decoder network in a similar fashion to approaches considered in deep learning for image-to-image regression tasks. Since normal neural networks are data intensive and cannot provide predictive uncertainty, we propose a Bayesian approach to convolutional neural nets. A recently introduced variational gradient descent algorithm based on Stein's method is scaled to deep convolutional networks to perform approximate Bayesian inference on millions of uncertain network parameters. This approach achieves state of the art performance in terms of predictive accuracy and uncertainty quantification in comparison to other approaches in Bayesian neural networks as well as techniques that include Gaussian processes and ensemble methods even when the training data size is relatively small. To evaluate the performance of this approach, we consider standard uncertainty quantification benchmark problems including flow in heterogeneous media defined in terms of limited data-driven permeability realizations. The performance of the surrogate model developed is very good even though there is no underlying structure shared between the input (permeability) and output (flow/pressure) fields as is often the case in the image-to-image regression models used in computer vision problems. Studies are performed with an underlying stochastic input dimensionality up to $4,225$ where most other uncertainty quantification methods fail. Uncertainty propagation tasks are considered and the predictive output Bayesian statistics are compared to those obtained with Monte Carlo estimates.

52 pages, 28 figures, submitted to Journal of Computational Physics

References in corpus (12)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Fully Convolutional Networks for Semantic Segmentation
• Depth Map Prediction from a Single Image using a Multi-Scale Deep Network
• Weight Uncertainty in Neural Networks
• Understanding deep learning requires rethinking generalization
• Understanding the Effective Receptive Field in Deep Convolutional Neural Networks
• Probabilistic Backpropagation for Scalable Learning of Bayesian Neural Networks
• A machine learning approach for efficient uncertainty quantification using multiscale methods
• Theory of Deep Learning III: explaining the non-overfitting puzzle
• A Hierarchical Bayesian Framework for Constructing Sparsity-inducing Priors
• Bayesian model and dimension reduction for uncertainty propagation: applications in random media

Cited by in corpus (143)

• Fourier Neural Operator for Parametric Partial Differential Equations
• Surrogate Modeling for Fluid Flows Based on Physics-Constrained Deep Learning Without Simulation Data
• Physics-Constrained Deep Learning for High-dimensional Surrogate Modeling and Uncertainty Quantification without Labeled Data
• Aleatoric uncertainty estimation with test-time augmentation for medical image segmentation with convolutional neural networks
• Machine learning in cardiovascular flows modeling: Predicting arterial blood pressure from non-invasive 4D flow MRI data using physics-informed neural networks
• Quantifying total uncertainty in physics-informed neural networks for solving forward and inverse stochastic problems
• Deep UQ: Learning deep neural network surrogate models for high dimensional uncertainty quantification
• A physics-informed variational DeepONet for predicting the crack path in brittle materials
• Deep Learning of Vortex Induced Vibrations
• A deep-learning-based surrogate model for data assimilation in dynamic subsurface flow problems
• Deep convolutional encoder-decoder networks for uncertainty quantification of dynamic multiphase flow in heterogeneous media
• Modeling the Dynamics of PDE Systems with Physics-Constrained Deep Auto-Regressive Networks
• Deep Learning of Subsurface Flow via Theory-guided Neural Network
• Data Driven Governing Equations Approximation Using Deep Neural Networks
• Deep autoregressive neural networks for high-dimensional inverse problems in groundwater contaminant source identification
• PhyCRNet: Physics-informed Convolutional-Recurrent Network for Solving Spatiotemporal PDEs
• Transfer learning based multi-fidelity physics informed deep neural network
• EikoNet: Solving the Eikonal equation with Deep Neural Networks
• Transformers for Modeling Physical Systems
• Multipole Graph Neural Operator for Parametric Partial Differential Equations
• Deep-learning-based surrogate flow modeling and geological parameterization for data assimilation in 3D subsurface flow
• Data-Driven Deep Learning of Partial Differential Equations in Modal Space
• Simulator-free Solution of High-Dimensional Stochastic Elliptic Partial Differential Equations using Deep Neural Networks
• Real-time high-resolution CO${}_2$ geological storage prediction using nested Fourier neural operators
• Neural Operator: Graph Kernel Network for Partial Differential Equations
• Integration of adversarial autoencoders with residual dense convolutional networks for estimation of non-Gaussian hydraulic conductivities
• Physics-Informed Probabilistic Learning of Linear Embeddings of Non-linear Dynamics With Guaranteed Stability
• D3M: A deep domain decomposition method for partial differential equations
• Quantifying model form uncertainty in Reynolds-averaged turbulence models with Bayesian deep neural networks
• The Random Feature Model for Input-Output Maps between Banach Spaces
• Multiphase flow prediction with deep neural networks
• Parametric generation of conditional geological realizations using generative neural networks
• Learning Deep Implicit Fourier Neural Operators (IFNOs) with Applications to Heterogeneous Material Modeling
• Improving prediction of the terrestrial water storage anomalies during the GRACE and GRACE-FO gap with Bayesian convolutional neural networks
• A mesh-free method for interface problems using the deep learning approach
• LaSDI: Parametric Latent Space Dynamics Identification
• Efficient Uncertainty Quantification for Dynamic Subsurface Flow with Surrogate by Theory-guided Neural Network
• Solving inverse problems using conditional invertible neural networks
• Surrogate and inverse modeling for two-phase flow in porous media via theory-guided convolutional neural network
• Efficient Deep Learning Techniques for Multiphase Flow Simulation in Heterogeneous Porous Media
• Normalizing field flows: Solving forward and inverse stochastic differential equations using physics-informed flow models
• Physics-Informed Generative Adversarial Networks for Stochastic Differential Equations
• Choose a Transformer: Fourier or Galerkin
• Data-driven soliton solutions and model parameters of nonlinear wave models via the conservation-law constrained neural network method
• On generalized residue network for deep learning of unknown dynamical systems
• Teaching the Incompressible Navier-Stokes Equations to Fast Neural Surrogate Models in 3D
• Mosaic Flows: A Transferable Deep Learning Framework for Solving PDEs on Unseen Domains
• Quantifying Uncertainty in Discrete-Continuous and Skewed Data with Bayesian Deep Learning
• Deep Learning Methods for Partial Differential Equations and Related Parameter Identification Problems
• Use of Multifidelity Training Data and Transfer Learning for Efficient Construction of Subsurface Flow Surrogate Models
• In-Context Operator Learning with Data Prompts for Differential Equation Problems
• Efficient physics-informed neural networks using hash encoding
• Adaptive deep density approximation for Fokker-Planck equations
• A Deep Learning approach to Reduced Order Modelling of Parameter Dependent Partial Differential Equations
• Automated quantification of myocardial tissue characteristics from native T1 mapping using neural networks with Bayesian inference for uncertainty-based quality-control
• Nonlocal Kernel Network (NKN): a Stable and Resolution-Independent Deep Neural Network
• Attention U-Net as a surrogate model for groundwater prediction
• A physics-aware, probabilistic machine learning framework for coarse-graining high-dimensional systems in the Small Data regime
• Manifold learning-based polynomial chaos expansions for high-dimensional surrogate models
• Deep reinforcement learning for optimal well control in subsurface systems with uncertain geology
• GINNs: Graph-Informed Neural Networks for Multiscale Physics
• What machine learning can do for computational solid mechanics
• Wasserstein Generative Adversarial Uncertainty Quantification in Physics-Informed Neural Networks
• Physics informed deep learning for computational elastodynamics without labeled data
• Deep Neural Helmholtz Operators for 3D Elastic Wave Propagation and Inversion
• Learning Large-scale Subsurface Simulations with a Hybrid Graph Network Simulator
• Fully probabilistic deep models for forward and inverse problems in parametric PDEs
• Designing Accurate Emulators for Scientific Processes using Calibration-Driven Deep Models
• Uncertainty-aware performance assessment of optical imaging modalities with invertible neural networks
• Bayesian multiscale deep generative model for the solution of high-dimensional inverse problems
• Bayesian model and dimension reduction for uncertainty propagation: applications in random media
• Extended Dynamic Mode Decomposition for Inhomogeneous Problems
• A Bi-fidelity Surrogate Modeling Approach for Uncertainty Propagation in Three-Dimensional Hemodynamic Simulations
• Deep encoder-decoder hierarchical convolutional neural networks for conjugate heat transfer surrogate modeling
• Adaptive physics-informed neural operator for coarse-grained non-equilibrium flows
• A probabilistic generative model for semi-supervised training of coarse-grained surrogates and enforcing physical constraints through virtual observables
• Deep convolutional neural networks for uncertainty propagation in random fields
• Continuous conditional generative adversarial networks for data-driven solutions of poroelasticity with heterogeneous material properties
• A hybrid data driven-physics constrained Gaussian process regression framework with deep kernel for uncertainty quantification
• Physics Informed Deep Learning for Transport in Porous Media. Buckley Leverett Problem
• Uncertainty Quantification and Propagation in Surrogate-based Bayesian Inference
• AdaSpring: Context-adaptive and Runtime-evolutionary Deep Model Compression for Mobile Applications
• Solving and Learning Nonlinear PDEs with Gaussian Processes
• Encoding Invariances in Deep Generative Models
• Bi-fidelity Variational Auto-encoder for Uncertainty Quantification
• Operator Learning Using Random Features: A Tool for Scientific Computing
• Stein variational gradient descent with local approximations
• Bayesian neural networks for weak solution of PDEs with uncertainty quantification
• A Bayesian Deep Learning Approach to Near-Term Climate Prediction
• Meta-Auto-Decoder for Solving Parametric Partial Differential Equations
• Comparison of Neural FEM and Neural Operator Methods for applications in Solid Mechanics
• opPINN: Physics-Informed Neural Network with operator learning to approximate solutions to the Fokker-Planck-Landau equation
• Solving PDE-constrained Control Problems Using Operator Learning
• Fast uncertainty quantification of reservoir simulation with variational U-Net
• Physics-Constrained Bayesian Neural Network for Fluid Flow Reconstruction with Sparse and Noisy Data
• Uncertainty quantification of two-phase flow in porous media via coupled-TgNN surrogate model
• LordNet: An Efficient Neural Network for Learning to Solve Parametric Partial Differential Equations without Simulated Data
• Deep-learning-based reduced-order modeling for subsurface flow simulation
• Physics-Aware Neural Implicit Solvers for multiscale, parametric PDEs with applications in heterogeneous media
• Quantum DeepONet: Neural operators accelerated by quantum computing
• An adaptive surrogate modeling based on deep neural networks for large-scale Bayesian inverse problems
• Deep learning-based super-resolution fluorescence microscopy on small datasets
• Deep adaptive sampling for surrogate modeling without labeled data
• A Deep Learning-Accelerated Data Assimilation and Forecasting Workflow for Commercial-Scale Geologic Carbon Storage
• Discovery of Dynamics Using Linear Multistep Methods
• Performance Analysis of Semi-supervised Learning in the Small-data Regime using VAEs
• Stochastic Modeling of Inhomogeneities in the Aortic Wall and Uncertainty Quantification using a Bayesian Encoder-Decoder Surrogate
• Bayesian Sparse learning with preconditioned stochastic gradient MCMC and its applications
• Stochastic Domain Decomposition Based on Variable-Separation Method
• A data-assisted two-stage method for the inverse random source problem
• Novel Deep neural networks for solving Bayesian statistical inverse
• Reduced-order Deep Learning for Flow Dynamics
• Physics Informed Deep Learning for Flow and Transport in Porous Media
• Learning to solve Bayesian inverse problems: An amortized variational inference approach using Gaussian and Flow guides
• Learning phase field mean curvature flows with neural networks
• Learning Incompressible Fluid Dynamics from Scratch -- Towards Fast, Differentiable Fluid Models that Generalize
• Cycle Consistency-based Uncertainty Quantification of Neural Networks in Inverse Imaging Problems
• Transfer Learning on Multi-Dimensional Data: A Novel Approach to Neural Network-Based Surrogate Modeling
• Out of distribution detection for intra-operative functional imaging
• A Bayesian Multiscale Deep Learning Framework for Flows in Random Media
• MCMC-Net: Accelerating Markov Chain Monte Carlo with Neural Networks for Inverse Problems
• Leveraging Interpolation Models and Error Bounds for Verifiable Scientific Machine Learning
• Automatic Segmentation of Organs-at-Risk from Head-and-Neck CT using Separable Convolutional Neural Network with Hard-Region-Weighted Loss
• Evaluating the Stability of Deep Learning Latent Feature Spaces
• Physics-Guided Discovery of Highly Nonlinear Parametric Partial Differential Equations
• Beyond Point Estimate: Inferring Ensemble Prediction Variation from Neuron Activation Strength in Recommender Systems
• Efficient Computation Reduction in Bayesian Neural Networks Through Feature Decomposition and Memorization
• Prediction of Discretization of GMsFEM using Deep Learning
• Uncertainty Quantification of Graph Convolution Neural Network Models of Evolving Processes
• yNet: a multi-input convolutional network for ultra-fast simulation of field evolvement
• Learning Partial Differential Equations in Reproducing Kernel Hilbert Spaces
• Deep learning based on mixed-variable physics informed neural network for solving fluid dynamics without simulation data
• Deep learning of parameterized equations with applications to uncertainty quantification
• Variational inference formulation for a model-free simulation of a dynamical system with unknown parameters by a recurrent neural network
• Bayesian Reasoning for Physics Informed Neural Networks
• Probabilistic partition of unity networks for high-dimensional regression problems
• Improved Surrogates in Inertial Confinement Fusion with Manifold and Cycle Consistencies
• Exploring the BSM parameter space with Neural Network aided Simulation-Based Inference
• An acceleration strategy for randomize-then-optimize sampling via deep neural networks
• Automatic Segmentation of Gross Target Volume of Nasopharynx Cancer using Ensemble of Multiscale Deep Neural Networks with Spatial Attention
• A survey of unsupervised learning methods for high-dimensional uncertainty quantification in black-box-type problems
• CompNO: A Novel Foundation Model approach for solving Partial Differential Equations
• CAZSL: Zero-Shot Regression for Pushing Models by Generalizing Through Context