SynthText3D: Synthesizing Scene Text Images from 3D Virtual Worlds

Abstract

With the development of deep neural networks, the demand for a significant amount of annotated training data becomes the performance bottlenecks in many fields of research and applications. Image synthesis can generate annotated images automatically and freely, which gains increasing attention recently. In this paper, we propose to synthesize scene text images from the 3D virtual worlds, where the precise descriptions of scenes, editable illumination/visibility, and realistic physics are provided. Different from the previous methods which paste the rendered text on static 2D images, our method can render the 3D virtual scene and text instances as an entirety. In this way, real-world variations, including complex perspective transformations, various illuminations, and occlusions, can be realized in our synthesized scene text images. Moreover, the same text instances with various viewpoints can be produced by randomly moving and rotating the virtual camera, which acts as human eyes. The experiments on the standard scene text detection benchmarks using the generated synthetic data demonstrate the effectiveness and superiority of the proposed method. The code and synthetic data is available at: https://github.com/MhLiao/SynthText3D

arXiv commentAccepted by SCIS

References in corpus (12)

• Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
• Learning from Synthetic Humans
• TextBoxes++: A Single-Shot Oriented Scene Text Detector
• Synthetic Data and Artificial Neural Networks for Natural Scene Text Recognition
• R2CNN: Rotational Region CNN for Orientation Robust Scene Text Detection
• TextBoxes: A Fast Text Detector with a Single Deep Neural Network
• Detecting Curve Text in the Wild: New Dataset and New Solution
• Learning Multilayer Channel Features for Pedestrian Detection
• SceneNet RGB-D: 5M Photorealistic Images of Synthetic Indoor Trajectories with Ground Truth
• Beyond Grand Theft Auto V for Training, Testing and Enhancing Deep Learning in Self Driving Cars
• Mask TextSpotter: An End-to-End Trainable Neural Network for Spotting Text with Arbitrary Shapes
• An Annotation Saved is an Annotation Earned: Using Fully Synthetic Training for Object Instance Detection

Cited by in corpus (3)

• Scene Text Detection and Recognition: The Deep Learning Era
• Rethinking Irregular Scene Text Recognition
• Self-Training for Domain Adaptive Scene Text Detection