Combining OCR Models for Reading Early Modern Printed Books

Abstract

In this paper, we investigate the usage of fine-grained font recognition on OCR for books printed from the 15th to the 18th century. We used a newly created dataset for OCR of early printed books for which fonts are labeled with bounding boxes. We know not only the font group used for each character, but the locations of font changes as well. In books of this period, we frequently find font group changes mid-line or even mid-word that indicate changes in language. We consider 8 different font groups present in our corpus and investigate 13 different subsets: the whole dataset and text lines with a single font, multiple fonts, Roman fonts, Gothic fonts, and each of the considered fonts, respectively. We show that OCR performance is strongly impacted by font style and that selecting fine-tuned models with font group recognition has a very positive impact on the results. Moreover, we developed a system using local font group recognition in order to combine the output of multiple font recognition models, and show that while slower, this approach performs better not only on text lines composed of multiple fonts but on the ones containing a single font only as well.

arXiv commentAccepted to ICDAR23

References in corpus (6)

• End-to-end Handwritten Paragraph Text Recognition Using a Vertical Attention Network
• TrOCR: Transformer-based Optical Character Recognition with Pre-trained Models
• Recurrence-free unconstrained handwritten text recognition using gated fully convolutional network
• Improving OCR Accuracy on Early Printed Books by utilizing Cross Fold Training and Voting
• MaskOCR: Text Recognition with Masked Encoder-Decoder Pretraining
• Rethinking Text Line Recognition Models

Cited by in corpus (2)

• Efficient Annotation of Medieval Charters
• Robust and Efficient Writer-Independent IMU-Based Handwriting Recognition