Recurrent Spatial Transformer Networks
arXiv:1509.05329
Abstract
We integrate the recently proposed spatial transformer network (SPN) [Jaderberg et. al 2015] into a recurrent neural network (RNN) to form an RNN-SPN model. We use the RNN-SPN to classify digits in cluttered MNIST sequences. The proposed model achieves a single digit error of 1.5% compared to 2.9% for a convolutional networks and 2.0% for convolutional networks with SPN layers. The SPN outputs a zoomed, rotated and skewed version of the input image. We investigate different down-sampling factors (ratio of pixel in input and output) for the SPN and show that the RNN-SPN model is able to down-sample the input images without deteriorating performance. The down-sampling in RNN-SPN can be thought of as adaptive down-sampling that minimizes the information loss in the regions of interest. We attribute the superior performance of the RNN-SPN to the fact that it can attend to a sequence of regions of interest.
References in corpus (5)
Cited by in corpus (10)
- Learning for Video Compression
- STN-OCR: A single Neural Network for Text Detection and Text Recognition
- Joint Line Segmentation and Transcription for End-to-End Handwritten Paragraph Recognition
- MERANet: Facial Micro-Expression Recognition using 3D Residual Attention Network
- MagicVO: End-to-End Monocular Visual Odometry through Deep Bi-directional Recurrent Convolutional Neural Network
- Probabilistic Spatial Transformer Networks
- A spatiotemporal model with visual attention for video classification
- A Biologically Inspired Visual Working Memory for Deep Networks
- StampNet: unsupervised multi-class object discovery
- Locally Smoothed Neural Networks