LAMBERT: Layout-Aware (Language) Modeling for information extraction

Abstract

We introduce a simple new approach to the problem of understanding documents where non-trivial layout influences the local semantics. To this end, we modify the Transformer encoder architecture in a way that allows it to use layout features obtained from an OCR system, without the need to re-learn language semantics from scratch. We only augment the input of the model with the coordinates of token bounding boxes, avoiding, in this way, the use of raw images. This leads to a layout-aware language model which can then be fine-tuned on downstream tasks. The model is evaluated on an end-to-end information extraction task using four publicly available datasets: Kleister NDA, Kleister Charity, SROIE and CORD. We show that our model achieves superior performance on datasets consisting of visually rich documents, while also outperforming the baseline RoBERTa on documents with flat layout (NDA \(F_{1}\) increase from 78.50 to 80.42). Our solution ranked first on the public leaderboard for the Key Information Extraction from the SROIE dataset, improving the SOTA \(F_{1}\)-score from 97.81 to 98.17.

arXiv commentaccepted to ICDAR 2021

References in corpus (6)

• Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
• Convolutional Sequence to Sequence Learning
• LayoutLM: Pre-training of Text and Layout for Document Image Understanding
• GPipe: Efficient Training of Giant Neural Networks using Pipeline Parallelism
• LayoutLMv2: Multi-modal Pre-training for Visually-Rich Document Understanding
• Kleister: A novel task for Information Extraction involving Long Documents with Complex Layout

Cited by in corpus (3)

• Document AI: Benchmarks, Models and Applications
• ViBERTgrid: A Jointly Trained Multi-Modal 2D Document Representation for Key Information Extraction from Documents
• Form2Seq : A Framework for Higher-Order Form Structure Extraction