Deformation Robust Roto-Scale-Translation Equivariant CNNs
arXiv:2111.10978
Abstract
Incorporating group symmetry directly into the learning process has proved to be an effective guideline for model design. By producing features that are guaranteed to transform covariantly to the group actions on the inputs, group-equivariant convolutional neural networks (G-CNNs) achieve significantly improved generalization performance in learning tasks with intrinsic symmetry. General theory and practical implementation of G-CNNs have been studied for planar images under either rotation or scaling transformation, but only individually. We present, in this paper, a roto-scale-translation equivariant CNN (RST-CNN), that is guaranteed to achieve equivariance jointly over these three groups via coupled group convolutions. Moreover, as symmetry transformations in reality are rarely perfect and typically subject to input deformation, we provide a stability analysis of the equivariance of representation to input distortion, which motivates the truncated expansion of the convolutional filters under (pre-fixed) low-frequency spatial modes. The resulting model provably achieves deformation-robust RST equivariance, i.e., the RST symmetry is still "approximately" preserved when the transformation is "contaminated" by a nuisance data deformation, a property that is especially important for out-of-distribution generalization. Numerical experiments on MNIST, Fashion-MNIST, and STL-10 demonstrate that the proposed model yields remarkable gains over prior arts, especially in the small data regime where both rotation and scaling variations are present within the data.
References in corpus (19)
- Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
- Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
- End to End Learning for Self-Driving Cars
- Improved Regularization of Convolutional Neural Networks with Cutout
- Group Equivariant Convolutional Networks
- Tensor field networks: Rotation- and translation-equivariant neural networks for 3D point clouds
- 3D Steerable CNNs: Learning Rotationally Equivariant Features in Volumetric Data
- Rotation equivariant vector field networks
- Convolutional Kernel Networks
- Universal Invariant and Equivariant Graph Neural Networks
- Locally Scale-Invariant Convolutional Neural Networks
- Scale-Invariant Convolutional Neural Networks
- General -Equivariant Steerable CNNs
- N-body Networks: a Covariant Hierarchical Neural Network Architecture for Learning Atomic Potentials
- Deep Scale-spaces: Equivariance Over Scale
- Scale equivariance in CNNs with vector fields
- DCFNet: Deep Neural Network with Decomposed Convolutional Filters
- DeepSphere: a graph-based spherical CNN
- Scaling-Translation-Equivariant Networks with Decomposed Convolutional Filters