Wasserstein Distributionally Robust Optimization: Theory and Applications in Machine Learning

Abstract

Many decision problems in science, engineering and economics are affected by uncertain parameters whose distribution is only indirectly observable through samples. The goal of data-driven decision-making is to learn a decision from finitely many training samples that will perform well on unseen test samples. This learning task is difficult even if all training and test samples are drawn from the same distribution -- especially if the dimension of the uncertainty is large relative to the training sample size. Wasserstein distributionally robust optimization seeks data-driven decisions that perform well under the most adverse distribution within a certain Wasserstein distance from a nominal distribution constructed from the training samples. In this tutorial we will argue that this approach has many conceptual and computational benefits. Most prominently, the optimal decisions can often be computed by solving tractable convex optimization problems, and they enjoy rigorous out-of-sample and asymptotic consistency guarantees. We will also show that Wasserstein distributionally robust optimization has interesting ramifications for statistical learning and motivates new approaches for fundamental learning tasks such as classification, regression, maximum likelihood estimation or minimum mean square error estimation, among others.

36 pages

References in corpus (9)

• Robust Wasserstein Profile Inference and Applications to Machine Learning
• Distributionally Robust Logistic Regression
• Wasserstein Discriminant Analysis
• A Sparse Multi-Scale Algorithm for Dense Optimal Transport
• Multilevel Clustering via Wasserstein Means
• Robust Hypothesis Testing Using Wasserstein Uncertainty Sets
• Principal Geodesic Analysis for Probability Measures under the Optimal Transport Metric
• Learning Probability Measures with respect to Optimal Transport Metrics
• Optimal Uncertainty Size in Distributionally Robust Inverse Covariance Estimation

Cited by in corpus (8)

• Geometric Dataset Distances via Optimal Transport
• GeneraLight: Improving Environment Generalization of Traffic Signal Control via Meta Reinforcement Learning
• A Distributionally Robust Area Under Curve Maximization Model
• Distributionally robust stochastic programs with side information based on trimmings -- Extended version
• Online Stochastic Convex Optimization: Wasserstein Distance Variation
• Robust Graph Learning Under Wasserstein Uncertainty
• Frank-Wolfe Methods in Probability Space
• Orthounimodal Distributionally Robust Optimization: Representation, Computation and Multivariate Extreme Event Applications