Scale equivariance in CNNs with vector fields

Abstract

We study the effect of injecting local scale equivariance into Convolutional Neural Networks. This is done by applying each convolutional filter at multiple scales. The output is a vector field encoding for the maximally activating scale and the scale itself, which is further processed by the following convolutional layers. This allows all the intermediate representations to be locally scale equivariant. We show that this improves the performance of the model by over $20\%$ in the scale equivariant task of regressing the scaling factor applied to randomly scaled MNIST digits. Furthermore, we find it also useful for scale invariant tasks, such as the actual classification of randomly scaled digits. This highlights the usefulness of allowing for a compact representation that can also learn relationships between different local scales by keeping internal scale equivariance.

arXiv commentICML/FAIM 2018 workshop on Towards learning with limited labels: Equivariance, Invariance, and Beyond (oral presentation)

References in corpus (6)

• Tensor field networks: Rotation- and translation-equivariant neural networks for 3D point clouds
• Equivariance Through Parameter-Sharing
• Locally Scale-Invariant Convolutional Neural Networks
• Scale-Invariant Convolutional Neural Networks
• Oriented Response Networks
• Learning Steerable Filters for Rotation Equivariant CNNs

Cited by in corpus (16)

• Deep Scale-spaces: Equivariance Over Scale
• Scale-Equivariant Steerable Networks
• Scale Steerable Filters for Locally Scale-Invariant Convolutional Neural Networks
• Scale-covariant and scale-invariant Gaussian derivative networks
• Attentive Group Equivariant Convolutional Networks
• Neural Networks Enforcing Physical Symmetries in Nonlinear Dynamical Lattices: The Case Example of the Ablowitz-Ladik Model
• Resolution learning in deep convolutional networks using scale-space theory
• Scale-invariant scale-channel networks: Deep networks that generalise to previously unseen scales
• Co-Attentive Equivariant Neural Networks: Focusing Equivariance On Transformations Co-Occurring In Data
• B-Spline CNNs on Lie Groups
• Scaling-Translation-Equivariant Networks with Decomposed Convolutional Filters
• Coordinate Independent Convolutional Networks -- Isometry and Gauge Equivariant Convolutions on Riemannian Manifolds
• Group Equivariant Stand-Alone Self-Attention For Vision
• Can we have it all? On the Trade-off between Spatial and Adversarial Robustness of Neural Networks
• Capacity of Group-invariant Linear Readouts from Equivariant Representations: How Many Objects can be Linearly Classified Under All Possible Views?
• Deformation Robust Roto-Scale-Translation Equivariant CNNs