Harmonic Networks: Deep Translation and Rotation Equivariance

Abstract

Translating or rotating an input image should not affect the results of many computer vision tasks. Convolutional neural networks (CNNs) are already translation equivariant: input image translations produce proportionate feature map translations. This is not the case for rotations. Global rotation equivariance is typically sought through data augmentation, but patch-wise equivariance is more difficult. We present Harmonic Networks or H-Nets, a CNN exhibiting equivariance to patch-wise translation and 360-rotation. We achieve this by replacing regular CNN filters with circular harmonics, returning a maximal response and orientation for every receptive field patch. H-Nets use a rich, parameter-efficient and low computational complexity representation, and we show that deep feature maps within the network encode complicated rotational invariants. We demonstrate that our layers are general enough to be used in conjunction with the latest architectures and techniques, such as deep supervision and batch normalization. We also achieve state-of-the-art classification on rotated-MNIST, and competitive results on other benchmark challenges.

Submitted to CVPR 2017

Cited by in corpus (22)

• Deformable Convolutional Networks
• Retinal vessel segmentation based on Fully Convolutional Neural Networks
• Land cover mapping at very high resolution with rotation equivariant CNNs: towards small yet accurate models
• Rotation equivariant vector field networks
• Deep Complex Networks
• Deep convolutional networks for quality assessment of protein folds
• Interaction-and-Aggregation Network for Person Re-identification
• Extracting gamma-ray information from images with convolutional neural network methods on simulated Cherenkov Telescope Array data
• Polar Transformer Networks
• Learning Steerable Filters for Rotation Equivariant CNNs
• Rotational 3D Texture Classification Using Group Equivariant CNNs
• Harmonic Networks: Integrating Spectral Information into CNNs
• Deep Rotation Equivariant Network
• Motion Equivariant Networks for Event Cameras with the Temporal Normalization Transform
• Equivariant Learning of Stochastic Fields: Gaussian Processes and Steerable Conditional Neural Processes
• Rotation-Invariant Autoencoders for Signals on Spheres
• Beyond Planar Symmetry: Modeling human perception of reflection and rotation symmetries in the wild
• Augmentation Inside the Network
• A Push-Pull Layer Improves Robustness of Convolutional Neural Networks
• On Universalized Adversarial and Invariant Perturbations
• Deep rank-based transposition-invariant distances on musical sequences
• Motion Equivariance OF Event-based Camera Data with the Temporal Normalization Transform