Generalized Resilience and Robust Statistics

Abstract

Robust statistics traditionally focuses on outliers, or perturbations in total variation distance. However, a dataset could be corrupted in many other ways, such as systematic measurement errors and missing covariates. We generalize the robust statistics approach to consider perturbations under any Wasserstein distance, and show that robust estimation is possible whenever a distribution's population statistics are robust under a certain family of friendly perturbations. This generalizes a property called resilience previously employed in the special case of mean estimation with outliers. We justify the generalized resilience property by showing that it holds under moment or hypercontractive conditions. Even in the total variation case, these subsume conditions in the literature for mean estimation, regression, and covariance estimation; the resulting analysis simplifies and sometimes improves these known results in both population limit and finite-sample rate. Our robust estimators are based on minimum distance (MD) functionals (Donoho and Liu, 1988), which project onto a set of distributions under a discrepancy related to the perturbation. We present two approaches for designing MD estimators with good finite-sample rates: weakening the discrepancy and expanding the set of distributions. We also present connections to Gao et al. (2019)'s recent analysis of generative adversarial networks for robust estimation.

References in corpus (20)

• Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning
• Poisoning Attacks against Support Vector Machines
• Generalizing to Unseen Domains via Adversarial Data Augmentation
• Sever: A Robust Meta-Algorithm for Stochastic Optimization
• Recent Advances in Algorithmic High-Dimensional Robust Statistics
• Being Robust (in High Dimensions) Can Be Practical
• Dimension-free PAC-Bayesian bounds for matrices, vectors, and linear least squares regression
• Efficient Algorithms for Outlier-Robust Regression
• High Dimensional Robust Sparse Regression
• Outlier-robust moment-estimation via sum-of-squares
• Robust Estimation and Generative Adversarial Nets
• A Unified Approach to Robust Mean Estimation
• Robust estimation via generalized quasi-gradients
• Faster Algorithms for High-Dimensional Robust Covariance Estimation
• Outlier Robust Mean Estimation with Subgaussian Rates via Stability
• Generative Adversarial Nets for Robust Scatter Estimation: A Proper Scoring Rule Perspective
• Robust Regression via Mutivariate Regression Depth
• How Hard Is Robust Mean Estimation?
• Confidence regions and minimax rates in outlier-robust estimation on the probability simplex
• Nearly Tight Bounds for Robust Proper Learning of Halfspaces with a Margin

Cited by in corpus (15)

• Recent Advances in Algorithmic High-Dimensional Robust Statistics
• Private Mean Estimation of Heavy-Tailed Distributions
• Robust estimation via generalized quasi-gradients
• Improved Rates for Differentially Private Stochastic Convex Optimization with Heavy-Tailed Data
• Robust and Differentially Private Mean Estimation
• Outlier Robust Mean Estimation with Subgaussian Rates via Stability
• DORO: Distributional and Outlier Robust Optimization
• Differential privacy and robust statistics in high dimensions
• When does the Tukey median work?
• On Robust Mean Estimation under Coordinate-level Corruption
• Robust Estimation for Random Graphs
• Robust Mean Estimation on Highly Incomplete Data with Arbitrary Outliers
• On the power of adaptivity in statistical adversaries
• Gradient-based Data Subversion Attack Against Binary Classifiers
• Robust W-GAN-Based Estimation Under Wasserstein Contamination