BERTgrid: Contextualized Embedding for 2D Document Representation and Understanding

Abstract

For understanding generic documents, information like font sizes, column layout, and generally the positioning of words may carry semantic information that is crucial for solving a downstream document intelligence task. Our novel BERTgrid, which is based on Chargrid by Katti et al. (2018), represents a document as a grid of contextualized word piece embedding vectors, thereby making its spatial structure and semantics accessible to the processing neural network. The contextualized embedding vectors are retrieved from a BERT language model. We use BERTgrid in combination with a fully convolutional network on a semantic instance segmentation task for extracting fields from invoices. We demonstrate its performance on tabulated line item and document header field extraction.

4 pages, accepted at the "Document Intelligence" workshop of 33rd Conference on Neural Information Processing Systems (NeurIPS 2019), Vancouver, Canada

References in corpus (3)

• Efficient Estimation of Word Representations in Vector Space
• Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
• CUTIE: Learning to Understand Documents with Convolutional Universal Text Information Extractor

Cited by in corpus (14)

• Combining Visual and Textual Features for Semantic Segmentation of Historical Newspapers
• Document AI: Benchmarks, Models and Applications
• A Survey of Deep Learning Approaches for OCR and Document Understanding
• Spatial Dependency Parsing for Semi-Structured Document Information Extraction
• TRIE: End-to-End Text Reading and Information Extraction for Document Understanding
• Improving Information Extraction on Business Documents with Specific Pre-Training Tasks
• Deep Learning based Key Information Extraction from Business Documents: Systematic Literature Review
• StrucTexT: Structured Text Understanding with Multi-Modal Transformers
• Efficient Document Image Classification Using Region-Based Graph Neural Network
• Long-Range Transformer Architectures for Document Understanding
• Key Information Extraction From Documents: Evaluation And Generator
• Cost-effective End-to-end Information Extraction for Semi-structured Document Images
• Data-Efficient Information Extraction from Form-Like Documents
• One-shot Key Information Extraction from Document with Deep Partial Graph Matching