MixUp as Directional Adversarial Training

Abstract

In this work, we explain the working mechanism of MixUp in terms of adversarial training. We introduce a new class of adversarial training schemes, which we refer to as directional adversarial training, or DAT. In a nutshell, a DAT scheme perturbs a training example in the direction of another example but keeps its original label as the training target. We prove that MixUp is equivalent to a special subclass of DAT, in that it has the same expected loss function and corresponds to the same optimization problem asymptotically. This understanding not only serves to explain the effectiveness of MixUp, but also reveals a more general family of MixUp schemes, which we call Untied MixUp. We prove that the family of Untied MixUp schemes is equivalent to the entire class of DAT schemes. We establish empirically the existence of Untied Mixup schemes which improve upon MixUp.

12 pages, 1 figure, submitted to NeurIPS 2019

References in corpus (4)

• mixup: Beyond Empirical Risk Minimization
• Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples
• Understanding Adversarial Training: Increasing Local Stability of Neural Nets through Robust Optimization
• Adversarial Training Methods for Semi-Supervised Text Classification

Cited by in corpus (7)

• FMix: Enhancing Mixed Sample Data Augmentation
• TGDM: Target Guided Dynamic Mixup for Cross-Domain Few-Shot Learning
• Adversarially Optimized Mixup for Robust Classification
• Midpoint Regularization: from High Uncertainty Training to Conservative Classification
• Adversarial Mixing Policy for Relaxing Locally Linear Constraints in Mixup
• Regularization via Adaptive Pairwise Label Smoothing
• A Diversity-Enhanced and Constraints-Relaxed Augmentation for Low-Resource Classification