Weak Supervision for Generating Pixel-Level Annotations in Scene Text Segmentation

Abstract

Providing pixel-level supervisions for scene text segmentation is inherently difficult and costly, so that only few small datasets are available for this task. To face the scarcity of training data, previous approaches based on Convolutional Neural Networks (CNNs) rely on the use of a synthetic dataset for pre-training. However, synthetic data cannot reproduce the complexity and variability of natural images. In this work, we propose to use a weakly supervised learning approach to reduce the domain-shift between synthetic and real data. Leveraging the bounding-box supervision of the COCO-Text and the MLT datasets, we generate weak pixel-level supervisions of real images. In particular, the COCO-Text-Segmentation (COCO_TS) and the MLT-Segmentation (MLT_S) datasets are created and released. These two datasets are used to train a CNN, the Segmentation Multiscale Attention Network (SMANet), which is specifically designed to face some peculiarities of the scene text segmentation task. The SMANet is trained end-to-end on the proposed datasets, and the experiments show that COCO_TS and MLT_S are a valid alternative to synthetic images, allowing to use only a fraction of the training samples and improving significantly the performances.

arXiv admin note: substantial text overlap with arXiv:1904.00818

References in corpus (15)

• Adam: A Method for Stochastic Optimization
• Rethinking Atrous Convolution for Semantic Image Segmentation
• YOLOv3: An Incremental Improvement
• Self-Attention Generative Adversarial Networks
• A Structured Self-attentive Sentence Embedding
• OCNet: Object Context Network for Scene Parsing
• Weakly- and Semi-Supervised Learning of a DCNN for Semantic Image Segmentation
• COCO-Text: Dataset and Benchmark for Text Detection and Recognition in Natural Images
• Scene Text Detection via Holistic, Multi-Channel Prediction
• Dual Attention Network for Scene Segmentation
• Synthetic Data for Text Localisation in Natural Images
• DiSAN: Directional Self-Attention Network for RNN/CNN-Free Language Understanding
• Single Shot Text Detector with Regional Attention
• Untangling Local and Global Deformations in Deep Convolutional Networks for Image Classification and Sliding Window Detection
• TextSnake: A Flexible Representation for Detecting Text of Arbitrary Shapes

Cited by in corpus (4)

• FETNet: Feature Erasing and Transferring Network for Scene Text Removal
• A multi-stage GAN for multi-organ chest X-ray image generation and segmentation
• Rethinking Text Segmentation: A Novel Dataset and A Text-Specific Refinement Approach
• Self-supervised Scene Text Segmentation with Object-centric Layered Representations Augmented by Text Regions