Inductive Visual Localisation: Factorised Training for Superior Generalisation
arXiv:1807.08179
Abstract
End-to-end trained Recurrent Neural Networks (RNNs) have been successfully applied to numerous problems that require processing sequences, such as image captioning, machine translation, and text recognition. However, RNNs often struggle to generalise to sequences longer than the ones encountered during training. In this work, we propose to optimise neural networks explicitly for induction. The idea is to first decompose the problem in a sequence of inductive steps and then to explicitly train the RNN to reproduce such steps. Generalisation is achieved as the RNN is not allowed to learn an arbitrary internal state; instead, it is tasked with mimicking the evolution of a valid state. In particular, the state is restricted to a spatial memory map that tracks parts of the input image which have been accounted for in previous steps. The RNN is trained for single inductive steps, where it produces updates to the memory in addition to the desired output. We evaluate our method on two different visual recognition problems involving visual sequences: (1) text spotting, i.e. joint localisation and reading of text in images containing multiple lines (or a block) of text, and (2) sequential counting of objects in aerial images. We show that inductive training of recurrent models enhances their generalisation ability on challenging image datasets.
arXiv commentIn BMVC 2018 (spotlight)
References in corpus (13)
- Sequence to Sequence Learning with Neural Networks
- ADADELTA: An Adaptive Learning Rate Method
- Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation
- Synthetic Data and Artificial Neural Networks for Natural Scene Text Recognition
- Multi-digit Number Recognition from Street View Imagery using Deep Convolutional Neural Networks
- Dilated Residual Networks
- Synthetic Data for Text Localisation in Natural Images
- End-to-End Text Recognition with Hybrid HMM Maxout Models
- Making Neural Programming Architectures Generalize via Recursion
- Recursive Recurrent Nets with Attention Modeling for OCR in the Wild
- Single Shot Text Detector with Regional Attention
- Joint Line Segmentation and Transcription for End-to-End Handwritten Paragraph Recognition
- FCN-rLSTM: Deep Spatio-Temporal Neural Networks for Vehicle Counting in City Cameras