Rotation equivariant vector field networks

Abstract

In many computer vision tasks, we expect a particular behavior of the output with respect to rotations of the input image. If this relationship is explicitly encoded, instead of treated as any other variation, the complexity of the problem is decreased, leading to a reduction in the size of the required model. In this paper, we propose the Rotation Equivariant Vector Field Networks (RotEqNet), a Convolutional Neural Network (CNN) architecture encoding rotation equivariance, invariance and covariance. Each convolutional filter is applied at multiple orientations and returns a vector field representing magnitude and angle of the highest scoring orientation at every spatial location. We develop a modified convolution operator relying on this representation to obtain deep architectures. We test RotEqNet on several problems requiring different responses with respect to the inputs' rotation: image classification, biomedical image segmentation, orientation estimation and patch matching. In all cases, we show that RotEqNet offers extremely compact models in terms of number of parameters and provides results in line to those of networks orders of magnitude larger.

10 pages, accepted at ICCV 2017

References in corpus (10)

• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Learning Deconvolution Network for Semantic Segmentation
• Learning to Compare Image Patches via Convolutional Neural Networks
• The Importance of Skip Connections in Biomedical Image Segmentation
• Learning Invariant Representations with Local Transformations
• When Face Recognition Meets with Deep Learning: an Evaluation of Convolutional Neural Networks for Face Recognition
• Oriented Response Networks
• Designing Deep Networks for Surface Normal Estimation
• Harmonic Networks: Deep Translation and Rotation Equivariance
• Steerable CNNs

Cited by in corpus (60)

• Land cover mapping at very high resolution with rotation equivariant CNNs: towards small yet accurate models
• On the Generalization of Equivariance and Convolution in Neural Networks to the Action of Compact Groups
• Gauge Equivariant Convolutional Networks and the Icosahedral CNN
• Spatio-Temporal Deep Learning-Based Undersampling Artefact Reduction for 2D Radial Cine MRI with Limited Data
• Generalizing Convolutional Neural Networks for Equivariance to Lie Groups on Arbitrary Continuous Data
• Oriented object detection in optical remote sensing images using deep learning: a survey
• Transformation-Invariant Network for Few-Shot Object Detection in Remote Sensing Images
• ReDet: A Rotation-equivariant Detector for Aerial Object Detection
• Scale Steerable Filters for Locally Scale-Invariant Convolutional Neural Networks
• A General Theory of Equivariant CNNs on Homogeneous Spaces
• Decision fusion with multiple spatial supports by conditional random fields
• Equivariant Transformer Networks
• Attentive Group Equivariant Convolutional Networks
• A Principled Design of Image Representation: Towards Forensic Tasks
• Neural Networks Enforcing Physical Symmetries in Nonlinear Dynamical Lattices: The Case Example of the Ablowitz-Ladik Model
• Learning Invariances in Neural Networks
• SE(3)-equivariant prediction of molecular wavefunctions and electronic densities
• A Practical Method for Constructing Equivariant Multilayer Perceptrons for Arbitrary Matrix Groups
• Rotation Equivariant Proximal Operator for Deep Unfolding Methods in Image Restoration
• Provably scale-covariant continuous hierarchical networks based on scale-normalized differential expressions coupled in cascade
• Constrained Learning with Non-Convex Losses
• Scale-invariant scale-channel networks: Deep networks that generalise to previously unseen scales
• Co-Attentive Equivariant Neural Networks: Focusing Equivariance On Transformations Co-Occurring In Data
• Approximately Equivariant Quantum Neural Network for $p4m$ Group Symmetries in Images
• B-Spline CNNs on Lie Groups
• Multitask Learning of Temporal Connectionism in Convolutional Networks using a Joint Distribution Loss Function to Simultaneously Identify Tools and Phase in Surgical Videos
• Rotation Equivariant Feature Image Pyramid Network for Object Detection in Optical Remote Sensing Imagery
• Scaling-Translation-Equivariant Networks with Decomposed Convolutional Filters
• Learning Steerable Filters for Rotation Equivariant CNNs
• Spin-Weighted Spherical CNNs
• Coordinate Independent Convolutional Networks -- Isometry and Gauge Equivariant Convolutions on Riemannian Manifolds
• Quaternion Equivariant Capsule Networks for 3D Point Clouds
• PreCM: The Padding-based Rotation Equivariant Convolution Mode for Semantic Segmentation
• Invariance-inducing regularization using worst-case transformations suffices to boost accuracy and spatial robustness
• Towards Learning Affine-Invariant Representations via Data-Efficient CNNs
• Deep Rotation Equivariant Network
• Equivariant neural networks and equivarification
• Countering Noisy Labels By Learning From Auxiliary Clean Labels
• Discrete Rotation Equivariance for Point Cloud Recognition
• Group Equivariant Stand-Alone Self-Attention For Vision
• The general theory of permutation equivarant neural networks and higher order graph variational encoders
• Learnable Gabor modulated complex-valued networks for orientation robustness
• Learning Compressed Transforms with Low Displacement Rank
• DCT-Conv: Coding filters in convolutional networks with Discrete Cosine Transform
• Can we have it all? On the Trade-off between Spatial and Adversarial Robustness of Neural Networks
• Deformation Robust Roto-Scale-Translation Equivariant CNNs
• Equivariant Networks for Pixelized Spheres
• Generating Unrestricted Adversarial Examples via Three Parameters
• Discriminative Learning of Similarity and Group Equivariant Representations
• Learning Equivariant Representations
• Self-Refining Deep Symmetry Enhanced Network for Rain Removal
• PREPRINT: Comparison of deep learning and hand crafted features for mining simulation data
• Training or Architecture? How to Incorporate Invariance in Neural Networks
• Equivariance-bridged SO(2)-Invariant Representation Learning using Graph Convolutional Network
• FILTRA: Rethinking Steerable CNN by Filter Transform
• Equivariant Point Network for 3D Point Cloud Analysis
• Rotating spiders and reflecting dogs: a class conditional approach to learning data augmentation distributions
• On Universalized Adversarial and Invariant Perturbations
• Geometric Data Augmentation Based on Feature Map Ensemble
• Rotational Rectification Network: Enabling Pedestrian Detection for Mobile Vision