The Geometry of Adversarial Training in Binary Classification

Abstract

We establish an equivalence between a family of adversarial training problems for non-parametric binary classification and a family of regularized risk minimization problems where the regularizer is a nonlocal perimeter functional. The resulting regularized risk minimization problems admit exact convex relaxations of the type $L^1+$ (nonlocal) $\mathrm{TV}$, a form frequently studied in image analysis and graph-based learning. A rich geometric structure is revealed by this reformulation which in turn allows us to establish a series of properties of optimal solutions of the original problem, including the existence of minimal and maximal solutions (interpreted in a suitable sense), and the existence of regular solutions (also interpreted in a suitable sense). In addition, we highlight how the connection between adversarial training and perimeter minimization problems provides a novel, directly interpretable, statistical motivation for a family of regularized risk minimization problems involving perimeter/total variation. The majority of our theoretical results are independent of the distance used to define adversarial attacks.

References in corpus (9)

• Kernel methods in machine learning
• Piecewise linear regularized solution paths
• Improving the Adversarial Robustness and Interpretability of Deep Neural Networks by Regularizing their Input Gradients
• Lower Bounds on Adversarial Robustness from Optimal Transport
• Consistency of Cheeger and Ratio Graph Cuts
• Distributionally Robust Learning
• Mixed Nash Equilibria in the Adversarial Examples Game
• The Many Faces of Adversarial Risk
• On the Existence of the Adversarial Bayes Classifier (Extended Version)

Cited by in corpus (3)

• A mean curvature flow arising in adversarial training
• Adversarial flows: A gradient flow characterization of adversarial attacks
• Adversarial Consistency and the Uniqueness of the Adversarial Bayes Classifier