Towards Unconstrained End-to-End Text Spotting

Abstract

We propose an end-to-end trainable network that can simultaneously detect and recognize text of arbitrary shape, making substantial progress on the open problem of reading scene text of irregular shape. We formulate arbitrary shape text detection as an instance segmentation problem; an attention model is then used to decode the textual content of each irregularly shaped text region without rectification. To extract useful irregularly shaped text instance features from image scale features, we propose a simple yet effective RoI masking step. Additionally, we show that predictions from an existing multi-step OCR engine can be leveraged as partially labeled training data, which leads to significant improvements in both the detection and recognition accuracy of our model. Our method surpasses the state-of-the-art for end-to-end recognition tasks on the ICDAR15 (straight) benchmark by 4.6%, and on the Total-Text (curved) benchmark by more than 16%.

arXiv commentAccepted to ICCV 2019 as oral presentation

References in corpus (9)

• Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
• Synthetic Data and Artificial Neural Networks for Natural Scene Text Recognition
• Focusing Attention: Towards Accurate Text Recognition in Natural Images
• R2CNN: Rotational Region CNN for Orientation Robust Scene Text Detection
• TextBoxes: A Fast Text Detector with a Single Deep Neural Network
• TextField: Learning A Deep Direction Field for Irregular Scene Text Detection
• Scene Text Detection and Recognition: The Deep Learning Era
• MSR: Multi-Scale Shape Regression for Scene Text Detection
• TextNet: Irregular Text Reading from Images with an End-to-End Trainable Network

Cited by in corpus (6)

• Enhancing Geometric Factors in Model Learning and Inference for Object Detection and Instance Segmentation
• Text Recognition in the Wild: A Survey
• FC2RN: A Fully Convolutional Corner Refinement Network for Accurate Multi-Oriented Scene Text Detection
• Exploring the Capacity of an Orderless Box Discretization Network for Multi-orientation Scene Text Detection
• ARTS: Eliminating Inconsistency between Text Detection and Recognition with Auto-Rectification Text Spotter
• A Low Computational Approach for Price Tag Recognition