An End-to-End Trainable Neural Network for Image-based Sequence Recognition and Its Application to Scene Text Recognition

Abstract

Image-based sequence recognition has been a long-standing research topic in computer vision. In this paper, we investigate the problem of scene text recognition, which is among the most important and challenging tasks in image-based sequence recognition. A novel neural network architecture, which integrates feature extraction, sequence modeling and transcription into a unified framework, is proposed. Compared with previous systems for scene text recognition, the proposed architecture possesses four distinctive properties: (1) It is end-to-end trainable, in contrast to most of the existing algorithms whose components are separately trained and tuned. (2) It naturally handles sequences in arbitrary lengths, involving no character segmentation or horizontal scale normalization. (3) It is not confined to any predefined lexicon and achieves remarkable performances in both lexicon-free and lexicon-based scene text recognition tasks. (4) It generates an effective yet much smaller model, which is more practical for real-world application scenarios. The experiments on standard benchmarks, including the IIIT-5K, Street View Text and ICDAR datasets, demonstrate the superiority of the proposed algorithm over the prior arts. Moreover, the proposed algorithm performs well in the task of image-based music score recognition, which evidently verifies the generality of it.

5 figures

References in corpus (7)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• ADADELTA: An Adaptive Learning Rate Method
• Synthetic Data and Artificial Neural Networks for Natural Scene Text Recognition
• Rich feature hierarchies for accurate object detection and semantic segmentation
• End-to-End Text Recognition with Hybrid HMM Maxout Models
• Deep Structured Output Learning for Unconstrained Text Recognition
• Supervised mid-level features for word image representation

Cited by in corpus (16)

• TextBoxes: A Fast Text Detector with a Single Deep Neural Network
• Towards End-to-End Car License Plates Detection and Recognition with Deep Neural Networks
• Towards End-to-end Text Spotting with Convolutional Recurrent Neural Networks
• ICDAR2017 Competition on Reading Chinese Text in the Wild (RCTW-17)
• Fully Convolutional Recurrent Network for Handwritten Chinese Text Recognition
• UTRNet: High-Resolution Urdu Text Recognition In Printed Documents
• A Binary Convolutional Encoder-decoder Network for Real-time Natural Scene Text Processing
• Unconstrained Scene Text and Video Text Recognition for Arabic Script
• Open Images V5 Text Annotation and Yet Another Mask Text Spotter
• Smart Library: Identifying Books in a Library using Richly Supervised Deep Scene Text Reading
• Learning to Read by Spelling: Towards Unsupervised Text Recognition
• Learning Spatial-Semantic Context with Fully Convolutional Recurrent Network for Online Handwritten Chinese Text Recognition
• Distantly Supervised Semantic Text Detection and Recognition for Broadcast Sports Videos Understanding
• A Compositional Textual Model for Recognition of Imperfect Word Images
• Extracting textual overlays from social media videos using neural networks
• SocialML: machine learning for social media video creators