Finite sample properties of parametric MMD estimation: robustness to misspecification and dependence

Abstract

Many works in statistics aim at designing a universal estimation procedure, that is, an estimator that would converge to the best approximation of the (unknown) data generating distribution in a model, without any assumption on this distribution. This question is of major interest, in particular because the universality property leads to the robustness of the estimator. In this paper, we tackle the problem of universal estimation using a minimum distance estimator presented in Briol et al. (2019) based on the Maximum Mean Discrepancy. We show that the estimator is robust to both dependence and to the presence of outliers in the dataset. Finally, we provide a theoretical study of the stochastic gradient descent algorithm used to compute the estimator, and we support our findings with numerical simulations. ** The proof of Proposition 4.4 in the published version contains a mistake. The mistake is fixed here (and the bound is actually improved by a factor 2). **

References in corpus (26)

• Variational Inference: A Review for Statisticians
• Kernel Mean Embedding of Distributions: A Review and Beyond
• Learning Decentralized Controllers for Robot Swarms with Graph Neural Networks
• Generative Moment Matching Networks
• Weakly dependent functional data
• InfoVAE: Information Maximizing Variational Autoencoders
• Geometric median and robust estimation in Banach spaces
• Hilbert space embeddings and metrics on probability measures
• Training generative neural networks via Maximum Mean Discrepancy optimization
• A Kernel Multiple Change-point Algorithm via Model Selection
• Kernel Belief Propagation
• Consistency of Variational Bayes Inference for Estimation and Model Selection in Mixtures
• A Linear-Time Kernel Goodness-of-Fit Test
• Statistical Inference for Generative Models with Maximum Mean Discrepancy
• MMD-Bayes: Robust Bayesian Estimation via Maximum Mean Discrepancy
• A new method for estimation and model selection: $\rho$-estimation
• Robust Estimation and Generative Adversarial Nets
• PAC-Bayes under potentially heavy tails
• Dimension-free PAC-Bayesian bounds for the estimation of the mean of a random vector
• Practical bounds on the error of Bayesian posterior approximations: A nonasymptotic approach
• Computationally Efficient Robust Estimation of Sparse Functionals
• Some Theoretical Properties of GANs
• Optimal Kullback-Leibler Aggregation in Mixture Density Estimation by Maximum Likelihood
• Robust subgaussian estimation of a mean vector in nearly linear time
• Distribution-robust mean estimation via smoothed random perturbations
• Minimax estimation of a p-dimensional linear functional in sparse Gaussian models and robust estimation of the mean

Cited by in corpus (4)

• Change Detection of Markov Kernels with Unknown Pre and Post Change Kernel
• Estimation of copulas via Maximum Mean Discrepancy
• Model-Free Change Point Detection for Mixing Processes
• Discrepancy-based Inference for Intractable Generative Models using Quasi-Monte Carlo