SVD Perspectives for Augmenting DeepONet Flexibility and Interpretability

Abstract

Deep operator networks (DeepONets) are powerful architectures for fast and accurate emulation of complex dynamics. As their remarkable generalization capabilities are primarily enabled by their projection-based attribute, we investigate connections with low-rank techniques derived from the singular value decomposition (SVD). We demonstrate that some of the concepts behind proper orthogonal decomposition (POD)-neural networks can improve DeepONet's design and training phases. These ideas lead us to a methodology extension that we name SVD-DeepONet. Moreover, through multiple SVD analyses, we find that DeepONet inherits from its projection-based attribute strong inefficiencies in representing dynamics characterized by symmetries. Inspired by the work on shifted-POD, we develop flexDeepONet, an architecture enhancement that relies on a pre-transformation network for generating a moving reference frame and isolating the rigid components of the dynamics. In this way, the physics can be represented on a latent space free from rotations, translations, and stretches, and an accurate projection can be performed to a low-dimensional basis. In addition to flexibility and interpretability, the proposed perspectives increase DeepONet's generalization capabilities and computational efficiencies. For instance, we show flexDeepONet can accurately surrogate the dynamics of 19 variables in a combustion chemistry application by relying on 95% less trainable parameters than the ones of the vanilla architecture. We argue that DeepONet and SVD-based methods can reciprocally benefit from each other. In particular, the flexibility of the former in leveraging multiple data sources and multifidelity knowledge in the form of both unstructured data and physics-informed constraints has the potential to greatly extend the applicability of methodologies such as POD and PCA.

References in corpus (23)

• Adam: A Method for Stochastic Optimization
• DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators
• Machine Learning for Fluid Mechanics
• DGM: A deep learning algorithm for solving partial differential equations
• Fourier Neural Operator for Parametric Partial Differential Equations
• A physics-informed variational DeepONet for predicting the crack path in brittle materials
• Lift & Learn: Physics-informed machine learning for large-scale nonlinear dynamical systems
• DeepM&Mnet: Inferring the electroconvection multiphysics fields based on operator approximation by neural networks
• A fast and accurate physics-informed neural network reduced order model with shallow masked autoencoder
• Operator learning for predicting multiscale bubble growth dynamics
• Learning Deep Implicit Fourier Neural Operators (IFNOs) with Applications to Heterogeneous Material Modeling
• A physics-informed operator regression framework for extracting data-driven continuum models
• Generating Random Earthquake Events for PTHA
• The Neural Network shifted-Proper Orthogonal Decomposition: a Machine Learning Approach for Non-linear Reduction of Hyperbolic Equations
• Learning Operators with Coupled Attention
• Simulating progressive intramural damage leading to aortic dissection using an operator-regression neural network
• General Tensor Decomposition, Moment Matrices and Applications
• On the influence of over-parameterization in manifold based surrogates and deep neural operators
• DeepONet prediction of linear instability waves in high-speed boundary layers
• Model Reduction with MapReduce-enabled Tall-and-Skinny Singular Value Decomposition
• Multiscale DeepONet for Nonlinear Operators in Oscillatory Function Spaces for Building Seismic Wave Responses
• Enhanced DeepONet for Modeling Partial Differential Operators Considering Multiple Input Functions
• MultiAuto-DeepONet: A Multi-resolution Autoencoder DeepONet for Nonlinear Dimension Reduction, Uncertainty Quantification and Operator Learning of Forward and Inverse Stochastic Problems

Cited by in corpus (8)

• Lagrangian PINNs: A causality-conforming solution to failure modes of physics-informed neural networks
• RiemannONets: Interpretable Neural Operators for Riemann Problems
• Adaptive physics-informed neural operator for coarse-grained non-equilibrium flows
• A Mathematical Guide to Operator Learning
• RandONet: Shallow-Networks with Random Projections for learning linear and nonlinear operators
• Estimating Ground Reaction Forces from Inertial Sensors
• Deciphering and integrating invariants for neural operator learning with various physical mechanisms
• Monte Carlo Neural PDE Solver for Learning PDEs via Probabilistic Representation