OrigamiNet: Weakly-Supervised, Segmentation-Free, One-Step, Full Page Text Recognition by learning to unfold

Abstract

Text recognition is a major computer vision task with a big set of associated challenges. One of those traditional challenges is the coupled nature of text recognition and segmentation. This problem has been progressively solved over the past decades, going from segmentation based recognition to segmentation free approaches, which proved more accurate and much cheaper to annotate data for. We take a step from segmentation-free single line recognition towards segmentation-free multi-line / full page recognition. We propose a novel and simple neural network module, termed \textbf{OrigamiNet}, that can augment any CTC-trained, fully convolutional single line text recognizer, to convert it into a multi-line version by providing the model with enough spatial capacity to be able to properly collapse a 2D input signal into 1D without losing information. Such modified networks can be trained using exactly their same simple original procedure, and using only \textbf{unsegmented} image and text pairs. We carry out a set of interpretability experiments that show that our trained models learn an accurate implicit line segmentation. We achieve state-of-the-art character error rate on both IAM \& ICDAR 2017 HTR benchmarks for handwriting recognition, surpassing all other methods in the literature. On IAM we even surpass single line methods that use accurate localization information during training. Our code is available online at \url{https://github.com/IntuitionMachines/OrigamiNet}.

Accepted to CVPR 2020

References in corpus (9)

• Adam: A Method for Stochastic Optimization
• Very Deep Convolutional Networks for Large-Scale Image Recognition
• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• Deep Residual Learning for Image Recognition
• Axiomatic Attribution for Deep Networks
• Fully Convolutional Networks for Semantic Segmentation
• SmoothGrad: removing noise by adding noise
• What Is Wrong With Scene Text Recognition Model Comparisons? Dataset and Model Analysis
• Joint Line Segmentation and Transcription for End-to-End Handwritten Paragraph Recognition

Cited by in corpus (5)

• Robust Text Line Detection in Historical Documents: Learning and Evaluation Methods
• Character-Based Handwritten Text Transcription with Attention Networks
• Faster DAN: Multi-target Queries with Document Positional Encoding for End-to-end Handwritten Document Recognition
• SPAN: a Simple Predict & Align Network for Handwritten Paragraph Recognition
• Robust End-to-End Offline Chinese Handwriting Text Page Spotter with Text Kernel