A Rigorous Theory of Conditional Mean Embeddings

Abstract

Conditional mean embeddings (CMEs) have proven themselves to be a powerful tool in many machine learning applications. They allow the efficient conditioning of probability distributions within the corresponding reproducing kernel Hilbert spaces (RKHSs) by providing a linear-algebraic relation for the kernel mean embeddings of the respective joint and conditional probability distributions. Both centred and uncentred covariance operators have been used to define CMEs in the existing literature. In this paper, we develop a mathematically rigorous theory for both variants, discuss the merits and problems of each, and significantly weaken the conditions for applicability of CMEs. In the course of this, we demonstrate a beautiful connection to Gaussian conditioning in Hilbert spaces.

arXiv comment30 pages, 3 figures

References in corpus (5)

• Kernel dimension reduction in regression
• Oblique projections and Schur complements
• A Measure-Theoretic Approach to Kernel Conditional Mean Embeddings
• Almost sure convergence of the largest and smallest eigenvalues of high-dimensional sample correlation matrices
• Singular Value Decomposition of Operators on Reproducing Kernel Hilbert Spaces

Cited by in corpus (12)

• Convergence Rates for Learning Linear Operators from Noisy Data
• A Measure-Theoretic Approach to Kernel Conditional Mean Embeddings
• Singular Value Decomposition of Operators on Reproducing Kernel Hilbert Spaces
• Kernel Partial Correlation Coefficient -- a Measure of Conditional Dependence
• The linear conditional expectation in Hilbert space
• When Invariant Representation Learning Meets Label Shift: Insufficiency and Theoretical Insights
• Kernel Autocovariance Operators of Stationary Processes: Estimation and Convergence
• Nonparametric approximation of conditional expectation operators
• Conditional Bures Metric for Domain Adaptation
• Consistent Long-Term Forecasting of Ergodic Dynamical Systems
• Sobolev Norm Learning Rates for Conditional Mean Embeddings
• Generalised Rank-Constrained Approximations of Hilbert-Schmidt Operators on Separable Hilbert Spaces and Applications