Robust Learning with Jacobian Regularization

Abstract

Design of reliable systems must guarantee stability against input perturbations. In machine learning, such guarantee entails preventing overfitting and ensuring robustness of models against corruption of input data. In order to maximize stability, we analyze and develop a computationally efficient implementation of Jacobian regularization that increases classification margins of neural networks. The stabilizing effect of the Jacobian regularizer leads to significant improvements in robustness, as measured against both random and adversarial input perturbations, without severely degrading generalization properties on clean data.

21 pages, 10 figures

References in corpus (12)

• Towards Deep Learning Models Resistant to Adversarial Attacks
• Integration with respect to the Haar measure on unitary, orthogonal and symplectic group
• Towards Deep Neural Network Architectures Robust to Adversarial Examples
• Sensitivity and Generalization in Neural Networks: an Empirical Study
• Improving the Adversarial Robustness and Interpretability of Deep Neural Networks by Regularizing their Input Gradients
• Robust Large Margin Deep Neural Networks
• Defensive Distillation is Not Robust to Adversarial Examples
• Spectral Norm Regularization for Improving the Generalizability of Deep Learning
• Motivating the Rules of the Game for Adversarial Example Research
• On some properties of orthogonal Weingarten functions
• Three Mechanisms of Weight Decay Regularization
• Unifying Adversarial Training Algorithms with Flexible Deep Data Gradient Regularization

Cited by in corpus (27)

• Uncertainty Estimation Using a Single Deep Deterministic Neural Network
• The Implicit and Explicit Regularization Effects of Dropout
• Understanding and Mitigating Exploding Inverses in Invertible Neural Networks
• Robust learning with implicit residual networks
• Fair Mixup: Fairness via Interpolation
• Gradient ${\ell }_1$ Regularization for Quantization Robustness
• Jacobian Regularization for Mitigating Universal Adversarial Perturbations
• DSRNA: Differentiable Search of Robust Neural Architectures
• Jacobian Adversarially Regularized Networks for Robustness
• On the relationship between class selectivity, dimensionality, and robustness
• Learning emergent PDEs in a learned emergent space
• Neuron with Steady Response Leads to Better Generalization
• On the human-recognizability phenomenon of adversarially trained deep image classifiers
• An Empirical Study on the Relation between Network Interpretability and Adversarial Robustness
• UniDrop: A Simple yet Effective Technique to Improve Transformer without Extra Cost
• Adversary Agnostic Robust Deep Reinforcement Learning
• Adversarial robustness via stochastic regularization of neural activation sensitivity
• Noisy Feature Mixup
• Robust Learning with Frequency Domain Regularization
• SVMax: A Feature Embedding Regularizer
• Improving Adversarial Robustness for Free with Snapshot Ensemble
• Local Disentanglement in Variational Auto-Encoders Using Jacobian $L_1$ Regularization
• The Geometric Occam's Razor Implicit in Deep Learning
• Model-Aware Regularization For Learning Approaches To Inverse Problems
• Linking average- and worst-case perturbation robustness via class selectivity and dimensionality
• Improving Resistance to Adversarial Deformations by Regularizing Gradients
• Likelihood Landscapes: A Unifying Principle Behind Many Adversarial Defenses