Recursive Recurrent Nets with Attention Modeling for OCR in the Wild

Abstract

We present recursive recurrent neural networks with attention modeling (R${}^2$AM) for lexicon-free optical character recognition in natural scene images. The primary advantages of the proposed method are: (1) use of recursive convolutional neural networks (CNNs), which allow for parametrically efficient and effective image feature extraction; (2) an implicitly learned character-level language model, embodied in a recurrent neural network which avoids the need to use N-grams; and (3) the use of a soft-attention mechanism, allowing the model to selectively exploit image features in a coordinated way, and allowing for end-to-end training within a standard backpropagation framework. We validate our method with state-of-the-art performance on challenging benchmark datasets: Street View Text, IIIT5k, ICDAR and Synth90k.

accepted at CVPR 2016

References in corpus (14)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Deep Learning in Neural Networks: An Overview
• Neural Machine Translation by Jointly Learning to Align and Translate
• Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
• Caffe: Convolutional Architecture for Fast Feature Embedding
• Visualizing and Understanding Recurrent Networks
• Synthetic Data and Artificial Neural Networks for Natural Scene Text Recognition
• Multi-digit Number Recognition from Street View Imagery using Deep Convolutional Neural Networks
• COCO-Text: Dataset and Benchmark for Text Detection and Recognition in Natural Images
• Show and Tell: A Neural Image Caption Generator
• Generalizing Pooling Functions in Convolutional Neural Networks: Mixed, Gated, and Tree
• End-to-End Text Recognition with Hybrid HMM Maxout Models
• Deep Structured Output Learning for Unconstrained Text Recognition
• Supervised mid-level features for word image representation

Cited by in corpus (21)

• Scene Text Recognition with Sliding Convolutional Character Models
• Reading Scene Text with Attention Convolutional Sequence Modeling
• Joint Line Segmentation and Transcription for End-to-End Handwritten Paragraph Recognition
• Scene Text Recognition from Two-Dimensional Perspective
• Scene Text Synthesis for Efficient and Effective Deep Network Training
• Edit Probability for Scene Text Recognition
• Transcribing Content from Structural Images with Spotlight Mechanism
• A Multi-Object Rectified Attention Network for Scene Text Recognition
• Recurrent Calibration Network for Irregular Text Recognition
• FOTS: Fast Oriented Text Spotting with a Unified Network
• Generating Text Sequence Images for Recognition
• Primitive Representation Learning for Scene Text Recognition
• STRIDE : Scene Text Recognition In-Device
• Learning to Read by Spelling: Towards Unsupervised Text Recognition
• AON: Towards Arbitrarily-Oriented Text Recognition
• TextNet: Irregular Text Reading from Images with an End-to-End Trainable Network
• Inductive Visual Localisation: Factorised Training for Superior Generalisation
• Simultaneous Recognition of Horizontal and Vertical Text in Natural Images
• Adaptive Text Recognition through Visual Matching
• Exploring Font-independent Features for Scene Text Recognition
• Using Human Psychophysics to Evaluate Generalization in Scene Text Recognition Models