Scale-covariant and scale-invariant Gaussian derivative networks

Abstract

This paper presents a hybrid approach between scale-space theory and deep learning, where a deep learning architecture is constructed by coupling parameterized scale-space operations in cascade. By sharing the learnt parameters between multiple scale channels, and by using the transformation properties of the scale-space primitives under scaling transformations, the resulting network becomes provably scale covariant. By in addition performing max pooling over the multiple scale channels, a resulting network architecture for image classification also becomes provably scale invariant. We investigate the performance of such networks on the MNISTLargeScale dataset, which contains rescaled images from original MNIST over a factor of 4 concerning training data and over a factor of 16 concerning testing data. It is demonstrated that the resulting approach allows for scale generalization, enabling good performance for classifying patterns at scales not present in the training data.

arXiv comment21 pages, 10 figures

References in corpus (17)

• Group Equivariant Convolutional Networks
• Gabor Convolutional Networks
• Deep Parametric Continuous Convolutional Neural Networks
• On the Generalization of Equivariance and Convolution in Neural Networks to the Action of Compact Groups
• Deep Scale-spaces: Equivariance Over Scale
• Scale equivariance in CNNs with vector fields
• Invariance of visual operations at the level of receptive fields
• Scale Steerable Filters for Locally Scale-Invariant Convolutional Neural Networks
• Understanding when spatial transformer networks do not support invariance, and what to do about it
• Provably scale-covariant continuous hierarchical networks based on scale-normalized differential expressions coupled in cascade
• Blurring the Line Between Structure and Learning to Optimize and Adapt Receptive Fields
• Dense scale selection over space, time and space-time
• Exploring the ability of CNNs to generalise to previously unseen scales over wide scale ranges
• PDO-eConvs: Partial Differential Operator Based Equivariant Convolutions
• From Discrete to Continuous Convolution Layers
• Provably scale-covariant networks from oriented quasi quadrature measures in cascade
• Inability of spatial transformations of CNN feature maps to support invariant recognition

Cited by in corpus (8)

• Discrete approximations of Gaussian smoothing and Gaussian derivatives
• Scale-invariant scale-channel networks: Deep networks that generalise to previously unseen scales
• Feature-Centered First Order Structure Tensor Scale-Space in 2D and 3D
• Covariance properties under natural image transformations for the generalized Gaussian derivative model for visual receptive fields
• Unified theory for joint covariance properties under geometric image transformations for spatio-temporal receptive fields according to the generalized Gaussian derivative model for visual receptive fields
• Covariant spatio-temporal receptive fields for spiking neural networks
• Scale generalisation properties of extended scale-covariant and scale-invariant Gaussian derivative networks on image datasets with spatial scaling variations
• Approximation properties relative to continuous scale space for hybrid discretizations of Gaussian derivative operators