Scale-covariant and scale-invariant Gaussian derivative networks

Abstract

This paper presents a hybrid approach between scale-space theory and deep learning, where a deep learning architecture is constructed by coupling parameterized scale-space operations in cascade. By sharing the learnt parameters between multiple scale channels, and by using the transformation properties of the scale-space primitives under scaling transformations, the resulting network becomes provably scale covariant. By in addition performing max pooling over the multiple scale channels, a resulting network architecture for image classification also becomes provably scale invariant. We investigate the performance of such networks on the MNISTLargeScale dataset, which contains rescaled images from original MNIST over a factor of 4 concerning training data and over a factor of 16 concerning testing data. It is demonstrated that the resulting approach allows for scale generalization, enabling good performance for classifying patterns at scales not present in the training data.

21 pages, 10 figures

References in corpus (9)

• Deep Parametric Continuous Convolutional Neural Networks
• Deep Scale-spaces: Equivariance Over Scale
• Scale Steerable Filters for Locally Scale-Invariant Convolutional Neural Networks
• Understanding when spatial transformer networks do not support invariance, and what to do about it
• Blurring the Line Between Structure and Learning to Optimize and Adapt Receptive Fields
• Exploring the ability of CNNs to generalise to previously unseen scales over wide scale ranges
• PDO-eConvs: Partial Differential Operator Based Equivariant Convolutions
• From Discrete to Continuous Convolution Layers
• Inability of spatial transformations of CNN feature maps to support invariant recognition

Cited by in corpus (8)

• Discrete approximations of Gaussian smoothing and Gaussian derivatives
• Scale-invariant scale-channel networks: Deep networks that generalise to previously unseen scales
• Feature-Centered First Order Structure Tensor Scale-Space in 2D and 3D
• Covariance properties under natural image transformations for the generalized Gaussian derivative model for visual receptive fields
• Unified theory for joint covariance properties under geometric image transformations for spatio-temporal receptive fields according to the generalized Gaussian derivative model for visual receptive fields
• Covariant spatio-temporal receptive fields for spiking neural networks
• Scale generalisation properties of extended scale-covariant and scale-invariant Gaussian derivative networks on image datasets with spatial scaling variations
• Approximation properties relative to continuous scale space for hybrid discretizations of Gaussian derivative operators