Convergence Rates for Learning Linear Operators from Noisy Data

Abstract

This paper studies the learning of linear operators between infinite-dimensional Hilbert spaces. The training data comprises pairs of random input vectors in a Hilbert space and their noisy images under an unknown self-adjoint linear operator. Assuming that the operator is diagonalizable in a known basis, this work solves the equivalent inverse problem of estimating the operator's eigenvalues given the data. Adopting a Bayesian approach, the theoretical analysis establishes posterior contraction rates in the infinite data limit with Gaussian priors that are not directly linked to the forward map of the inverse problem. The main results also include learning-theoretic generalization error guarantees for a wide range of distribution shifts. These convergence rates quantify the effects of data smoothness and true eigenvalue decay or growth, for compact or unbounded operators, respectively, on sample complexity. Numerical evidence supports the theory in diagonal and non-diagonal settings.

arXiv commentTo appear in SIAM/ASA Journal on Uncertainty Quantification (JUQ); 34 pages, 5 figures, 2 tables

References in corpus (29)

• Discovering governing equations from data: Sparse identification of nonlinear dynamical systems
• DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators
• Prediction in functional linear regression
• Data-driven approximation of the Koopman generator: Model reduction, system identification, and control
• A New Approach to Collaborative Filtering: Operator Estimation with Spectral Regularization
• Bayesian inverse problems with Gaussian priors
• MAP Estimators and Their Consistency in Bayesian Nonparametric Inverse Problems
• The Random Feature Model for Input-Output Maps between Banach Spaces
• Can stable and accurate neural networks be computed? -- On the barriers of deep learning and Smale's 18th problem
• Solving Electrical Impedance Tomography with Deep Learning
• Eigendecompositions of Transfer Operators in Reproducing Kernel Hilbert Spaces
• A physics-informed operator regression framework for extracting data-driven continuum models
• Posterior Contraction Rates for the Bayesian Approach to Linear Ill-Posed Inverse Problems
• Bayesian inverse problems with non-conjugate priors
• Learning elliptic partial differential equations with randomized linear algebra
• Bayesian Posterior Contraction Rates for Linear Severely Ill-posed Inverse Problems
• A Rigorous Theory of Conditional Mean Embeddings
• Data driven regularization by projection
• Derivative-Informed Projected Neural Networks for High-Dimensional Parametric Maps Governed by PDEs
• Convergence analysis of Tikhonov regularization for non-linear statistical inverse learning problems
• Bayesian linear inverse problems in regularity scales
• Conditional mean embeddings as regressors - supplementary
• Bayesian inverse problems with unknown operators
• Bayesian inference of an uncertain generalized diffusion operator
• Functional Linear Regression with Mixed Predictors
• Deep Neural Networks Are Effective At Learning High-Dimensional Hilbert-Valued Functions From Limited Data
• Designing truncated priors for direct and inverse Bayesian problems
• Nonparametric approximation of conditional expectation operators
• Learning Schatten--von Neumann Operators

Cited by in corpus (10)

• A Mathematical Guide to Operator Learning
• Operator Learning Using Random Features: A Tool for Scientific Computing
• MCMC-Net: Accelerating Markov Chain Monte Carlo with Neural Networks for Inverse Problems
• Transfer Learning on Multi-Dimensional Data: A Novel Approach to Neural Network-Based Surrogate Modeling
• Forward and backward problems for coupled subdiffusion systems
• Fast Macroscopic Forcing Method
• Learning Partial Differential Equations in Reproducing Kernel Hilbert Spaces
• Learning a Gaussian Mixture for Sparsity Regularization in Inverse Problems
• Covariance Operator Estimation via Adaptive Thresholding
• Generalised Rank-Constrained Approximations of Hilbert-Schmidt Operators on Separable Hilbert Spaces and Applications