On the optimal estimation of probability measures in weak and strong topologies

Abstract

Given random samples drawn i.i.d. from a probability measure $\mathbb{P}$ (defined on say, ${\mathbb{R}}^d$), it is well-known that the empirical estimator is an optimal estimator of $\mathbb{P}$ in weak topology but not even a consistent estimator of its density (if it exists) in the strong topology (induced by the total variation distance). On the other hand, various popular density estimators such as kernel and wavelet density estimators are optimal in the strong topology in the sense of achieving the minimax rate over all estimators for a Sobolev ball of densities. Recently, it has been shown in a series of papers by Giné and Nickl that these density estimators on $\mathbb{R}$ that are optimal in strong topology are also optimal in $\Vert \cdot {\Vert }_{\mathcal{F}}$ for certain choices of $\mathcal{F}$ such that $\Vert \cdot {\Vert }_{\mathcal{F}}$ metrizes the weak topology, where $\Vert \mathbb{P}{\Vert }_{\mathcal{F}}:=\sup \{\int f\mathrm{d}\mathbb{P}:f\in \mathcal{F}\}$. In this paper, we investigate this problem of optimal estimation in weak and strong topologies by choosing $\mathcal{F}$ to be a unit ball in a reproducing kernel Hilbert space (say ${\mathcal{F}}_H$ defined over ${\mathbb{R}}^d$), where this choice is both of theoretical and computational interest. Under some mild conditions on the reproducing kernel, we show that $\Vert \cdot {\Vert }_{{\mathcal{F}}_H}$ metrizes the weak topology and the kernel density estimator (with $L^1$ optimal bandwidth) estimates $\mathbb{P}$ at dimension independent optimal rate of $n^{-1/2}$ in $\Vert \cdot {\Vert }_{{\mathcal{F}}_H}$ along with providing a uniform central limit theorem for the kernel density estimator.

Published at http://dx.doi.org/10.3150/15-BEJ713 in the Bernoulli (http://isi.cbs.nl/bernoulli/) by the International Statistical Institute/Bernoulli Society (http://isi.cbs.nl/BS/bshome.htm)

References in corpus (4)

• Equivalence of distance-based and RKHS-based statistics in hypothesis testing
• A Kernel Method for the Two-Sample Problem
• Universality, Characteristic Kernels and RKHS Embedding of Measures
• Uniform limit theorems for wavelet density estimators

Cited by in corpus (20)

• Kernel Mean Embedding of Distributions: A Review and Beyond
• Learning Decentralized Controllers for Robot Swarms with Graph Neural Networks
• Multi-fidelity Bayesian Optimisation with Continuous Approximations
• Measuring Sample Quality with Kernels
• Statistical Inference for Generative Models with Maximum Mean Discrepancy
• Kernel Distribution Embeddings: Universal Kernels, Characteristic Kernels and Kernel Metrics on Distributions
• Metrizing Weak Convergence with Maximum Mean Discrepancies
• TraDE: Transformers for Density Estimation
• Minimax Estimation of Kernel Mean Embeddings
• A uniform kernel trick for high-dimensional two-sample problems
• Robust Sample Weighting to Facilitate Individualized Treatment Rule Learning for a Target Population
• On Distance and Kernel Measures of Conditional Independence
• KALE Flow: A Relaxed KL Gradient Flow for Probabilities with Disjoint Support
• Universal Hypothesis Testing with Kernels: Asymptotically Optimal Tests for Goodness of Fit
• Using Kernel-Based Statistical Distance to Study the Dynamics of Charged Particle Beams in Particle-Based Simulation Codes
• Trimmed sample means for robust uniform mean estimation and regression
• The GaussianSketch for Almost Relative Error Kernel Distance
• Asymptotically Optimal One- and Two-Sample Testing with Kernels
• Exponentially Consistent Kernel Two-Sample Tests
• MMD-based Variable Importance for Distributional Random Forest