DAN: a Segmentation-free Document Attention Network for Handwritten Document Recognition

Abstract

Unconstrained handwritten text recognition is a challenging computer vision task. It is traditionally handled by a two-step approach, combining line segmentation followed by text line recognition. For the first time, we propose an end-to-end segmentation-free architecture for the task of handwritten document recognition: the Document Attention Network. In addition to text recognition, the model is trained to label text parts using begin and end tags in an XML-like fashion. This model is made up of an FCN encoder for feature extraction and a stack of transformer decoder layers for a recurrent token-by-token prediction process. It takes whole text documents as input and sequentially outputs characters, as well as logical layout tokens. Contrary to the existing segmentation-based approaches, the model is trained without using any segmentation label. We achieve competitive results on the READ 2016 dataset at page level, as well as double-page level with a CER of 3.43% and 3.70%, respectively. We also provide results for the RIMES 2009 dataset at page level, reaching 4.54% of CER. We provide all source code and pre-trained model weights at https://github.com/FactoDeepLearning/DAN.

References in corpus (6)

• End-to-end Handwritten Paragraph Text Recognition Using a Vertical Attention Network
• Full Page Handwriting Recognition via Image to Sequence Extraction
• Multiple Document Datasets Pre-training Improves Text Line Detection With Deep Neural Networks
• Robust Text Line Detection in Historical Documents: Learning and Evaluation Methods
• Recurrence-free unconstrained handwritten text recognition using gated fully convolutional network
• Have convolutions already made recurrence obsolete for unconstrained handwritten text recognition ?

Cited by in corpus (9)

• HTR-VT: Handwritten Text Recognition with Vision Transformer
• Large Scale Genealogical Information Extraction From Handwritten Quebec Parish Records
• Faster DAN: Multi-target Queries with Document Positional Encoding for End-to-end Handwritten Document Recognition
• SelfDocSeg: A Self-Supervised vision-based Approach towards Document Segmentation
• DANIEL: A fast Document Attention Network for Information Extraction and Labelling of handwritten documents
• Self-Supervised Learning for Text Recognition: A Critical Survey
• Normalized vs Diplomatic Annotation: A Case Study of Automatic Information Extraction from Handwritten Uruguayan Birth Certificates
• Meta-DAN: towards an efficient prediction strategy for page-level handwritten text recognition
• Relaxed syntax modeling in Transformers for future-proof license plate recognition