Towards End-to-end Text Spotting with Convolutional Recurrent Neural Networks

Abstract

In this work, we jointly address the problem of text detection and recognition in natural scene images based on convolutional recurrent neural networks. We propose a unified network that simultaneously localizes and recognizes text with a single forward pass, avoiding intermediate processes like image cropping and feature re-calculation, word separation, or character grouping. In contrast to existing approaches that consider text detection and recognition as two distinct tasks and tackle them one by one, the proposed framework settles these two tasks concurrently. The whole framework can be trained end-to-end, requiring only images, the ground-truth bounding boxes and text labels. Through end-to-end training, the learned features can be more informative, which improves the overall performance. The convolutional features are calculated only once and shared by both detection and recognition, which saves processing time. Our proposed method has achieved competitive performance on several benchmark datasets.

arXiv comment14 pages

References in corpus (6)

• Adam: A Method for Stochastic Optimization
• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
• Robust Text Detection in Natural Scene Images
• TextBoxes: A Fast Text Detector with a Single Deep Neural Network
• An End-to-End Trainable Neural Network for Image-based Sequence Recognition and Its Application to Scene Text Recognition

Cited by in corpus (12)

• Multi-Oriented Scene Text Detection via Corner Localization and Region Segmentation
• Rotation-Sensitive Regression for Oriented Scene Text Detection
• FOTS: Fast Oriented Text Spotting with a Unified Network
• All You Need Is Boundary: Toward Arbitrary-Shaped Text Spotting
• Text Perceptron: Towards End-to-End Arbitrary-Shaped Text Spotting
• TextNet: Irregular Text Reading from Images with an End-to-End Trainable Network
• Detecting Text in the Wild with Deep Character Embedding Network
• A Novel Integrated Framework for Learning both Text Detection and Recognition
• AE TextSpotter: Learning Visual and Linguistic Representation for Ambiguous Text Spotting
• Single Shot Scene Text Retrieval
• TextContourNet: a Flexible and Effective Framework for Improving Scene Text Detection Architecture with a Multi-task Cascade
• A Machine Learning Framework for Data Ingestion in Document Images