Deep Scale-spaces: Equivariance Over Scale

Abstract

We introduce deep scale-spaces (DSS), a generalization of convolutional neural networks, exploiting the scale symmetry structure of conventional image recognition tasks. Put plainly, the class of an image is invariant to the scale at which it is viewed. We construct scale equivariant cross-correlations based on a principled extension of convolutions, grounded in the theory of scale-spaces and semigroups. As a very basic operation, these cross-correlations can be used in almost any modern deep learning architecture in a plug-and-play manner. We demonstrate our networks on the Patch Camelyon and Cityscapes datasets, to prove their utility and perform introspective studies to further understand their properties.

References in corpus (4)

• Group Equivariant Convolutional Networks
• Tensor field networks: Rotation- and translation-equivariant neural networks for 3D point clouds
• Scale equivariance in CNNs with vector fields
• A General Theory of Equivariant CNNs on Homogeneous Spaces

Cited by in corpus (34)

• SE(3)-Transformers: 3D Roto-Translation Equivariant Attention Networks
• Incorporating Symmetry into Deep Dynamics Models for Improved Generalization
• MDP Homomorphic Networks: Group Symmetries in Reinforcement Learning
• Gauge Equivariant Mesh CNNs: Anisotropic convolutions on geometric graphs
• Scale-Equivariant Steerable Networks
• A General Theory of Equivariant CNNs on Homogeneous Spaces
• Scale-covariant and scale-invariant Gaussian derivative networks
• Attentive Group Equivariant Convolutional Networks
• Neural Networks Enforcing Physical Symmetries in Nonlinear Dynamical Lattices: The Case Example of the Ablowitz-Ladik Model
• Meta-Learning Symmetries by Reparameterization
• Self-supervised Scale Equivariant Network for Weakly Supervised Semantic Segmentation
• A Wigner-Eckart Theorem for Group Equivariant Convolution Kernels
• Scale-invariant scale-channel networks: Deep networks that generalise to previously unseen scales
• Dilated convolution with learnable spacings
• Co-Attentive Equivariant Neural Networks: Focusing Equivariance On Transformations Co-Occurring In Data
• B-Spline CNNs on Lie Groups
• A Data and Compute Efficient Design for Limited-Resources Deep Learning
• Scaling-Translation-Equivariant Networks with Decomposed Convolutional Filters
• Spin-Weighted Spherical CNNs
• Feature-Centered First Order Structure Tensor Scale-Space in 2D and 3D
• Topographic VAEs learn Equivariant Capsules
• Covariance properties under natural image transformations for the generalized Gaussian derivative model for visual receptive fields
• Group Equivariant Stand-Alone Self-Attention For Vision
• Scale-Equivariant Imaging: Self-Supervised Learning for Image Super-Resolution and Deblurring
• Affine Self Convolution
• Implicit Equivariance in Convolutional Networks
• Capacity of Group-invariant Linear Readouts from Equivariant Representations: How Many Objects can be Linearly Classified Under All Possible Views?
• Deformation Robust Roto-Scale-Translation Equivariant CNNs
• Equivariant neural networks for inverse problems
• Can we have it all? On the Trade-off between Spatial and Adversarial Robustness of Neural Networks
• Scale Equivariant Neural Networks with Morphological Scale-Spaces
• Autoequivariant Network Search via Group Decomposition
• Wiggling Weights to Improve the Robustness of Classifiers
• Exploiting Redundancy: Separable Group Convolutional Networks on Lie Groups