The Geometry of Adversarial Training in Binary Classification

Abstract

We establish an equivalence between a family of adversarial training problems for non-parametric binary classification and a family of regularized risk minimization problems where the regularizer is a nonlocal perimeter functional. The resulting regularized risk minimization problems admit exact convex relaxations of the type $L^1+$ (nonlocal) $\mathrm{TV}$, a form frequently studied in image analysis and graph-based learning. A rich geometric structure is revealed by this reformulation which in turn allows us to establish a series of properties of optimal solutions of the original problem, including the existence of minimal and maximal solutions (interpreted in a suitable sense), and the existence of regular solutions (also interpreted in a suitable sense). In addition, we highlight how the connection between adversarial training and perimeter minimization problems provides a novel, directly interpretable, statistical motivation for a family of regularized risk minimization problems involving perimeter/total variation. The majority of our theoretical results are independent of the distance used to define adversarial attacks.

References in corpus (21)

• Kernel methods in machine learning
• Square-Root Lasso: Pivotal Recovery of Sparse Signals via Conic Programming
• The solution path of the generalized lasso
• Piecewise linear regularized solution paths
• Improving the Adversarial Robustness and Interpretability of Deep Neural Networks by Regularizing their Input Gradients
• Robust Wasserstein Profile Inference and Applications to Machine Learning
• Continuum limit of total variation on point clouds
• Nonlocal curvature flows
• Lower Bounds on Adversarial Robustness from Optimal Transport
• Consistency of Cheeger and Ratio Graph Cuts
• Improved robustness to adversarial examples using Lipschitz regularization of the loss
• CLIP: Cheap Lipschitz Training of Neural Networks
• The Total Variation Flow in Metric Random Walk Spaces
• Learned convex regularizers for inverse problems
• Adversarially Robust Training through Structured Gradient Regularization
• Solution Paths of Variational Regularization Methods for Inverse Problems
• Distributionally Robust Learning
• Adversarial Classification: Necessary conditions and geometric flows
• Mixed Nash Equilibria in the Adversarial Examples Game
• The Many Faces of Adversarial Risk
• On the Existence of the Adversarial Bayes Classifier (Extended Version)

Cited by in corpus (3)

• A mean curvature flow arising in adversarial training
• Adversarial flows: A gradient flow characterization of adversarial attacks
• Adversarial Consistency and the Uniqueness of the Adversarial Bayes Classifier