Adversarial Robustness via Fisher-Rao Regularization

Abstract

Adversarial robustness has become a topic of growing interest in machine learning since it was observed that neural networks tend to be brittle. We propose an information-geometric formulation of adversarial defense and introduce FIRE, a new Fisher-Rao regularization for the categorical cross-entropy loss, which is based on the geodesic distance between the softmax outputs corresponding to natural and perturbed input features. Based on the information-geometric properties of the class of softmax distributions, we derive an explicit characterization of the Fisher-Rao Distance (FRD) for the binary and multiclass cases, and draw some interesting properties as well as connections with standard regularization metrics. Furthermore, for a simple linear and Gaussian model, we show that all Pareto-optimal points in the accuracy-robustness region can be reached by FIRE while other state-of-the-art methods fail. Empirically, we evaluate the performance of various classifiers trained with the proposed loss on standard datasets, showing up to a simultaneous 1\% of improvement in terms of clean and robust performances while reducing the training time by 20\% over the best-performing methods.

arXiv commentIEEE Transactions on Pattern Analysis and Machine Intelligence (Early Access)

References in corpus (19)

• Distilling the Knowledge in a Neural Network
• Explaining and Harnessing Adversarial Examples
• Intriguing properties of neural networks
• Transferability in Machine Learning: from Phenomena to Black-Box Attacks using Adversarial Samples
• Theoretically Principled Trade-off between Robustness and Accuracy
• Provable defenses against adversarial examples via the convex outer adversarial polytope
• Certified Adversarial Robustness via Randomized Smoothing
• Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks
• Certified Defenses against Adversarial Examples
• Using Pre-Training Can Improve Model Robustness and Uncertainty
• Detecting Adversarial Samples from Artifacts
• On the (Statistical) Detection of Adversarial Examples
• Adversarial Weight Perturbation Helps Robust Generalization
• Certified Adversarial Robustness with Additive Noise
• Are Labels Required for Improving Adversarial Robustness?
• Scaling provable adversarial defenses
• Provable Robustness of ReLU networks via Maximization of Linear Regions
• Controlling Neural Level Sets
• Adversarial Spheres

Cited by in corpus (6)

• On Closed-Form Expressions for the Fisher-Rao Distance
• A numerical approximation method for the Fisher-Rao distance between multivariate normal distributions
• The Fisher-Rao Loss for Learning under Label Noise
• Rainproof: An Umbrella To Shield Text Generators From Out-Of-Distribution Data
• Transductive Learning for Textual Few-Shot Classification in API-based Embedding Models
• Adversarial attacks on neural networks through canonical Riemannian foliations