Kernel Mean Shrinkage Estimators

Abstract

A mean function in a reproducing kernel Hilbert space (RKHS), or a kernel mean, is central to kernel methods in that it is used by many classical algorithms such as kernel principal component analysis, and it also forms the core inference step of modern kernel methods that rely on embedding probability distributions in RKHSs. Given a finite sample, an empirical average has been used commonly as a standard estimator of the true kernel mean. Despite a widespread use of this estimator, we show that it can be improved thanks to the well-known Stein phenomenon. We propose a new family of estimators called kernel mean shrinkage estimators (KMSEs), which benefit from both theoretical justifications and good empirical performance. The results demonstrate that the proposed estimators outperform the standard one, especially in a "large d, small n" paradigm.

41 pages

References in corpus (4)

• Entropy inference and the James-Stein estimator, with application to nonlinear gene association networks
• A Kernel Method for the Two-Sample Problem
• Density Estimation in Infinite Dimensional Exponential Families
• Kernel Belief Propagation

Cited by in corpus (8)

• Kernel Mean Embedding of Distributions: A Review and Beyond
• Learning Decentralized Controllers for Robot Swarms with Graph Neural Networks
• Gaussian Processes and Kernel Methods: A Review on Connections and Equivalences
• Bayesian Learning of Kernel Embeddings
• MONK -- Outlier-Robust Mean Embedding Estimation by Median-of-Means
• Minimax Estimation of Kernel Mean Embeddings
• Strictly Proper Kernel Scoring Rules and Divergences with an Application to Kernel Two-Sample Hypothesis Testing
• Deconditional Downscaling with Gaussian Processes