Lipschitz Properties for Deep Convolutional Networks

Abstract

In this paper we discuss the stability properties of convolutional neural networks. Convolutional neural networks are widely used in machine learning. In classification they are mainly used as feature extractors. Ideally, we expect similar features when the inputs are from the same class. That is, we hope to see a small change in the feature vector with respect to a deformation on the input signal. This can be established mathematically, and the key step is to derive the Lipschitz properties. Further, we establish that the stability results can be extended for more general networks. We give a formula for computing the Lipschitz bound, and compare it with other methods to show it is closer to the optimal value.

25 pages, 10 figures

References in corpus (1)

• Intriguing properties of neural networks

Cited by in corpus (7)

• Efficient and Accurate Estimation of Lipschitz Constants for Deep Neural Networks
• Lipschitz Bounds and Provably Robust Training by Laplacian Smoothing
• Towards the Quantification of Safety Risks in Deep Neural Networks
• Reinforcement Learning with Function-Valued Action Spaces for Partial Differential Equation Control
• Tune smarter not harder: A principled approach to tuning learning rates for shallow nets
• On Lipschitz Bounds of General Convolutional Neural Networks
• Energy Propagation in Deep Convolutional Neural Networks