Minimax Statistical Learning with Wasserstein Distances

Abstract

As opposed to standard empirical risk minimization (ERM), distributionally robust optimization aims to minimize the worst-case risk over a larger ambiguity set containing the original empirical distribution of the training data. In this work, we describe a minimax framework for statistical learning with ambiguity sets given by balls in Wasserstein space. In particular, we prove generalization bounds that involve the covering number properties of the original ERM problem. As an illustrative example, we provide generalization guarantees for transport-based domain adaptation problems where the Wasserstein distance between the source and target domain distributions can be reliably estimated from unlabeled samples.

published as a conference paper at NIPS 2018, change in title

Cited by in corpus (42)

• Distributionally Robust Optimization: A Review
• On the Convergence and Robustness of Adversarial Training
• On Learning Invariant Representation for Domain Adaptation
• Domain Adaptation with Conditional Distribution Matching and Generalized Label Shift
• Training individually fair ML models with Sensitive Subspace Robustness
• Invariant Risk Minimization Games
• Poisoning Attack against Estimating from Pairwise Comparisons
• Randomization matters. How to defend against strong adversarial attacks
• Device Heterogeneity in Federated Learning: A Superquantile Approach
• Federated Learning with Superquantile Aggregation for Heterogeneous Data
• Learning Bounds for Risk-sensitive Learning
• Estimating Generalization under Distribution Shifts via Domain-Invariant Representations
• Auditing ML Models for Individual Bias and Unfairness
• Improved OOD Generalization via Adversarial Training and Pre-training
• Learning to Learn Single Domain Generalization
• Butterfly: One-step Approach towards Wildly Unsupervised Domain Adaptation
• Interior-Point Methods Strike Back: Solving the Wasserstein Barycenter Problem
• Generalization of Reinforcement Learning with Policy-Aware Adversarial Data Augmentation
• Fast Epigraphical Projection-based Incremental Algorithms for Wasserstein Distributionally Robust Support Vector Machine
• Minimax Classification with 0-1 Loss and Performance Guarantees
• Improving Robustness using Generated Data
• Linear Regression Games: Convergence Guarantees to Approximate Out-of-Distribution Solutions
• An Optimal Transport View on Generalization
• Towards Theoretical Understandings of Robust Markov Decision Processes: Sample Complexity and Asymptotics
• Continuity of Generalized Entropy and Statistical Learning
• Individually Fair Ranking
• Tree-Sliced Variants of Wasserstein Distances
• k-GANs: Ensemble of Generative Models with Semi-Discrete Optimal Transport
• Mixed Nash Equilibria in the Adversarial Examples Game
• Coverage-Guaranteed Prediction Sets for Out-of-Distribution Data
• Wasserstein Distributionally Robust Inverse Multiobjective Optimization
• Adversarial Sample Enhanced Domain Adaptation: A Case Study on Predictive Modeling with Electronic Health Records
• Robust Deep Learning as Optimal Control: Insights and Convergence Guarantees
• Robust Unsupervised Learning via L-Statistic Minimization
• Supervised classification via minimax probabilistic transformations
• Principled learning method for Wasserstein distributionally robust optimization with local perturbations
• Group-Structured Adversarial Training
• Individually Fair Gradient Boosting
• The Role of Embedding Complexity in Domain-invariant Representations
• Counterfactual Supervision-based Information Bottleneck for Out-of-Distribution Generalization
• Teacher-Student Competition for Unsupervised Domain Adaptation
• Robust Machine Learning via Privacy/Rate-Distortion Theory