TRIE: End-to-End Text Reading and Information Extraction for Document Understanding
arXiv:2005.13118
Abstract
Since real-world ubiquitous documents (e.g., invoices, tickets, resumes and leaflets) contain rich information, automatic document image understanding has become a hot topic. Most existing works decouple the problem into two separate tasks, (1) text reading for detecting and recognizing texts in images and (2) information extraction for analyzing and extracting key elements from previously extracted plain text. However, they mainly focus on improving information extraction task, while neglecting the fact that text reading and information extraction are mutually correlated. In this paper, we propose a unified end-to-end text reading and information extraction network, where the two tasks can reinforce each other. Specifically, the multimodal visual and textual features of text reading are fused for information extraction and in turn, the semantics in information extraction contribute to the optimization of text reading. On three real-world datasets with diverse document images (from fixed layout to variable layout, from structured text to semi-structured text), our proposed method significantly outperforms the state-of-the-art methods in both efficiency and accuracy.
Accepted to ACM MM2020. Code is available at https://davar-lab.github.io/publication.html or https://github.com/hikopensource/DAVAR-Lab-OCR
References in corpus (14)
- PyTorch: An Imperative Style, High-Performance Deep Learning Library
- Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
- ADADELTA: An Adaptive Learning Rate Method
- VisualBERT: A Simple and Performant Baseline for Vision and Language
- Focusing Attention: Towards Accurate Text Recognition in Natural Images
- A Survey on Recent Advances in Named Entity Recognition from Deep Learning models
- TextBoxes: A Fast Text Detector with a Single Deep Neural Network
- Rosetta: Large scale system for text detection and recognition in images
- ICDAR2019 Competition on Scanned Receipt OCR and Information Extraction
- BERTgrid: Contextualized Embedding for 2D Document Representation and Understanding
- Single Shot Text Detector with Regional Attention
- CUTIE: Learning to Understand Documents with Convolutional Universal Text Information Extractor
- Chargrid: Towards Understanding 2D Documents
- Text Perceptron: Towards End-to-End Arbitrary-Shaped Text Spotting