Robust Estimation and Generative Adversarial Nets

Abstract

Robust estimation under Huber's $\epsilon$-contamination model has become an important topic in statistics and theoretical computer science. Statistically optimal procedures such as Tukey's median and other estimators based on depth functions are impractical because of their computational intractability. In this paper, we establish an intriguing connection between $f$-GANs and various depth functions through the lens of $f$-Learning. Similar to the derivation of $f$-GANs, we show that these depth functions that lead to statistically optimal robust estimators can all be viewed as variational lower bounds of the total variation distance in the framework of $f$-Learning. This connection opens the door of computing robust estimators using tools developed for training GANs. In particular, we show in both theory and experiments that some appropriate structures of discriminator networks with hidden layers in GANs lead to statistically optimal robust location estimators for both Gaussian distribution and general elliptical distributions where first moment may not exist.

References in corpus (15)

• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• Spectral Normalization for Generative Adversarial Networks
• Estimating divergence functionals and the likelihood ratio by convex risk minimization
• Generative Moment Matching Networks
• Demystifying MMD GANs
• Sever: A Robust Meta-Algorithm for Stochastic Optimization
• Being Robust (in High Dimensions) Can Be Practical
• Understanding GANs: the LQG Setting
• Approximability of Discriminators Implies Diversity in GANs
• McGan: Mean and Covariance Feature Matching GAN
• Robust Learning of Fixed-Structure Bayesian Networks
• Computationally Efficient Robust Estimation of Sparse Functionals
• Robust Regression via Mutivariate Regression Depth
• On general notions of depth for regression
• Halfspace depths for scatter, concentration and shape matrices

Cited by in corpus (11)

• A Selective Overview of Deep Learning
• Limits of Deepfake Detection: A Robust Estimation Viewpoint
• Generalized Resilience and Robust Statistics
• Finite sample properties of parametric MMD estimation: robustness to misspecification and dependence
• When OT meets MoM: Robust estimation of Wasserstein Distance
• Generative Adversarial Nets for Robust Scatter Estimation: A Proper Scoring Rule Perspective
• Adversarial Robustness of Stabilized NeuralODEs Might be from Obfuscated Gradients
• StrokeGAN: Reducing Mode Collapse in Chinese Font Generation via Stroke Encoding
• Deconstructing Generative Adversarial Networks
• Depth Descent Synchronization in $\mathrm{SO}(D)$
• Robust W-GAN-Based Estimation Under Wasserstein Contamination