Robust Neural Networks inspired by Strong Stability Preserving Runge-Kutta methods
arXiv:2010.10047
Abstract
Deep neural networks have achieved state-of-the-art performance in a variety of fields. Recent works observe that a class of widely used neural networks can be viewed as the Euler method of numerical discretization. From the numerical discretization perspective, Strong Stability Preserving (SSP) methods are more advanced techniques than the explicit Euler method that produce both accurate and stable solutions. Motivated by the SSP property and a generalized Runge-Kutta method, we propose Strong Stability Preserving networks (SSP networks) which improve robustness against adversarial attacks. We empirically demonstrate that the proposed networks improve the robustness against adversarial examples without any defensive methods. Further, the SSP networks are complementary with a state-of-the-art adversarial training scheme. Lastly, our experiments show that SSP networks suppress the blow-up of adversarial perturbations. Our results open up a way to study robust architectures of neural networks leveraging rich knowledge from numerical discretization literature.
arXiv commentECCV 2020
References in corpus (8)
- Intriguing properties of neural networks
- Theoretically Principled Trade-off between Robustness and Accuracy
- Provable defenses against adversarial examples via the convex outer adversarial polytope
- Robustness and Regularization of Support Vector Machines
- On the Convergence and Robustness of Adversarial Training
- Latent ODEs for Irregularly-Sampled Time Series
- FFJORD: Free-form Continuous Dynamics for Scalable Reversible Generative Models
- NAIS-Net: Stable Deep Networks from Non-Autonomous Differential Equations