Deep Neural Network for Semantic-based Text Recognition in Images
arXiv:1908.01403
Abstract
State-of-the-art text spotting systems typically aim to detect isolated words or word-by-word text in images of natural scenes and ignore the semantic coherence within a region of text. However, when interpreted together, seemingly isolated words may be easier to recognize. On this basis, we propose a novel "semantic-based text recognition" (STR) deep learning model that reads text in images with the help of understanding context. STR consists of several modules. We introduce the Text Grouping and Arranging (TGA) algorithm to connect and order isolated text regions. A text-recognition network interprets isolated words. Benefiting from semantic information, a sequenceto-sequence network model efficiently corrects inaccurate and uncertain phrases produced earlier in the STR pipeline. We present experiments on two new distinct datasets that contain scanned catalog images of interior designs and photographs of protesters with hand-written signs, respectively. Our results show that our STR model outperforms a baseline method that uses state-of-the-art single-wordrecognition techniques on both datasets. STR yields a high accuracy rate of 90% on the catalog images and 71% on the more difficult protest images, suggesting its generality in recognizing text.
References in corpus (11)
- Sequence to Sequence Learning with Neural Networks
- Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
- A Neural Conversational Model
- Synthetic Data and Artificial Neural Networks for Natural Scene Text Recognition
- Effective Approaches to Attention-based Neural Machine Translation
- Grammar as a Foreign Language
- Text-Attentional Convolutional Neural Networks for Scene Text Detection
- On Using Very Large Target Vocabulary for Neural Machine Translation
- Neural Networks for Text Correction and Completion in Keyboard Decoding
- Global Encoding for Abstractive Summarization
- Attention-based Encoder-Decoder Networks for Spelling and Grammatical Error Correction