Group-Structured Adversarial Training

Abstract

Robust training methods against perturbations to the input data have received great attention in the machine learning literature. A standard approach in this direction is adversarial training which learns a model using adversarially-perturbed training samples. However, adversarial training performs suboptimally against perturbations structured across samples such as universal and group-sparse shifts that are commonly present in biological data such as gene expression levels of different tissues. In this work, we seek to close this optimality gap and introduce Group-Structured Adversarial Training (GSAT) which learns a model robust to perturbations structured across samples. We formulate GSAT as a non-convex concave minimax optimization problem which minimizes a group-structured optimal transport cost. Specifically, we focus on the applications of GSAT for group-sparse and rank-constrained perturbations modeled using group and nuclear norm penalties. In order to solve GSAT's non-smooth optimization problem in those cases, we propose a new minimax optimization algorithm called GDADMM by combining Gradient Descent Ascent (GDA) and Alternating Direction Method of Multipliers (ADMM). We present several applications of the GSAT framework to gain robustness against structured perturbations for image recognition and computational biology datasets.

References in corpus (17)

• TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
• Explaining and Harnessing Adversarial Examples
• Guaranteed Minimum-Rank Solutions of Linear Matrix Equations via Nuclear Norm Minimization
• Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)
• Provable defenses against adversarial examples via the convex outer adversarial polytope
• Certified Defenses against Adversarial Examples
• Adversarial Feature Selection against Evasion Attacks
• Defensive Distillation is Not Robust to Adversarial Examples
• Distributionally Robust Logistic Regression
• Regularization via Mass Transportation
• Structured Adversarial Attack: Towards General Implementation and Better Interpretability
• Attacking the Madry Defense Model with $L_1$-based Adversarial Examples
• Certified Defenses for Adversarial Patches
• Minimax Statistical Learning with Wasserstein Distances
• Breaking certified defenses: Semantic adversarial examples with spoofed robustness certificates
• Improved Image Wasserstein Attacks and Defenses
• Second-Order Provable Defenses against Adversarial Attacks