Generalization bounds for deep convolutional neural networks

Abstract

We prove bounds on the generalization error of convolutional networks. The bounds are in terms of the training loss, the number of parameters, the Lipschitz constant of the loss and the distance from the weights to the initial weights. They are independent of the number of pixels in the input, and the height and width of hidden feature maps. We present experiments using CIFAR-10 with varying hyperparameters of a deep convolutional network, comparing our bounds with practical generalization gaps.

arXiv commentPublished as a conference paper at ICLR 2020

Cited by in corpus (13)

• Generalization Bounds of Stochastic Gradient Descent for Wide and Deep Neural Networks
• Deep Learning Through the Lens of Example Difficulty
• How does Weight Correlation Affect the Generalisation Ability of Deep Neural Networks
• Information-Theoretic Generalization Bounds for Stochastic Gradient Descent
• Fantastic Four: Differentiable Bounds on Singular Values of Convolution Layers
• Finding the Needle in the Haystack with Convolutions: on the benefits of architectural bias
• Asymptotic Singular Value Distribution of Linear Convolutional Layers
• An Optimization and Generalization Analysis for Max-Pooling Networks
• Trap of Feature Diversity in the Learning of MLPs
• Relative Deviation Margin Bounds
• Learning with Multiclass AUC: Theory and Algorithms
• Compression Implies Generalization
• Skew Orthogonal Convolutions