Inverse Compositional Spatial Transformer Networks

Abstract

In this paper, we establish a theoretical connection between the classical Lucas & Kanade (LK) algorithm and the emerging topic of Spatial Transformer Networks (STNs). STNs are of interest to the vision and learning communities due to their natural ability to combine alignment and classification within the same theoretical framework. Inspired by the Inverse Compositional (IC) variant of the LK algorithm, we present Inverse Compositional Spatial Transformer Networks (IC-STNs). We demonstrate that IC-STNs can achieve better performance than conventional STNs with less model capacity; in particular, we show superior performance in pure image alignment tasks as well as joint alignment/classification problems on real-world problems.

References in corpus (3)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Caffe: Convolutional Architecture for Fast Feature Embedding
• Identity Mappings in Deep Residual Networks

Cited by in corpus (10)

• Deformable Convolutional Networks
• ST-GAN: Spatial Transformer Generative Adversarial Networks for Image Compositing
• Iterative Transformer Network for 3D Point Cloud
• LS-Net: Learning to Solve Nonlinear Least Squares for Monocular Stereo
• Explicit Disentanglement of Appearance and Perspective in Generative Models
• CyLKs: Unsupervised Cycle Lucas-Kanade Network for Landmark Tracking
• A spatiotemporal model with visual attention for video classification
• Geo-Supervised Visual Depth Prediction
• Long Short-Term Memory Spatial Transformer Network
• Efficient Subpixel Refinement with Symbolic Linear Predictors