Synthesis in Style: Semantic Segmentation of Historical Documents using Synthetic Data

Abstract

One of the most pressing problems in the automated analysis of historical documents is the availability of annotated training data. The problem is that labeling samples is a time-consuming task because it requires human expertise and thus, cannot be automated well. In this work, we propose a novel method to construct synthetic labeled datasets for historical documents where no annotations are available. We train a StyleGAN model to synthesize document images that capture the core features of the original documents. While originally, the StyleGAN architecture was not intended to produce labels, it indirectly learns the underlying semantics to generate realistic images. Using our approach, we can extract the semantic information from the intermediate feature maps and use it to generate ground truth labels. To investigate if our synthetic dataset can be used to segment the text in historical documents, we use it to train multiple supervised segmentation models and evaluate their performance. We also train these models on another dataset created by a state-of-the-art synthesis approach to show that the models trained on our dataset achieve better results while requiring even less human annotation effort.

arXiv commentCode available at: https://github.com/hendraet/synthesis-in-style

References in corpus (9)

• TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation
• ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness
• dhSegment: A generic deep-learning approach for document segmentation
• UnrealText: Synthesizing Realistic Scene Text Images from the Unreal World
• Multiple Document Datasets Pre-training Improves Text Line Detection With Deep Neural Networks
• Open Evaluation Tool for Layout Analysis of Document Images
• HORAE: an annotated dataset of books of hours
• Two-stage generative adversarial networks for document image binarization with color noise and background removal
• One Model to Reconstruct Them All: A Novel Way to Use the Stochastic Noise in StyleGAN