Analysis of Invariance and Robustness via Invertibility of ReLU-Networks
arXiv:1806.09730
Abstract
Studying the invertibility of deep neural networks (DNNs) provides a principled approach to better understand the behavior of these powerful models. Despite being a promising diagnostic tool, a consistent theory on their invertibility is still lacking. We derive a theoretically motivated approach to explore the preimages of ReLU-layers and mechanisms affecting the stability of the inverse. Using the developed theory, we numerically show how this approach uncovers characteristic properties of the network.
References in corpus (12)
- Methods for Interpreting and Understanding Deep Neural Networks
- DGM: A deep learning algorithm for solving partial differential equations
- On the Number of Linear Regions of Deep Neural Networks
- The Loss Surfaces of Multilayer Networks
- Understanding Deep Convolutional Networks
- The Reversible Residual Network: Backpropagation Without Storing Activations
- Parseval Networks: Improving Robustness to Adversarial Examples
- Regularizing CNNs with Locally Constrained Decorrelations
- Mathematics of Deep Learning
- i-RevNet: Deep Invertible Networks
- Why are deep nets reversible: A simple theory, with implications for training
- Signal Recovery from Pooling Representations
Cited by in corpus (5)
- Excessive Invariance Causes Adversarial Vulnerability
- Exploiting Excessive Invariance caused by Norm-Bounded Adversarial Robustness
- Understanding and Mitigating Exploding Inverses in Invertible Neural Networks
- Training Invertible Neural Networks as Autoencoders
- Translating Diffusion, Wavelets, and Regularisation into Residual Networks