Joint Line Segmentation and Transcription for End-to-End Handwritten Paragraph Recognition

Abstract

Offline handwriting recognition systems require cropped text line images for both training and recognition. On the one hand, the annotation of position and transcript at line level is costly to obtain. On the other hand, automatic line segmentation algorithms are prone to errors, compromising the subsequent recognition. In this paper, we propose a modification of the popular and efficient multi-dimensional long short-term memory recurrent neural networks (MDLSTM-RNNs) to enable end-to-end processing of handwritten paragraphs. More particularly, we replace the collapse layer transforming the two-dimensional representation into a sequence of predictions by a recurrent version which can recognize one line at a time. In the proposed model, a neural network performs a kind of implicit line segmentation by computing attention weights on the image representation. The experiments on paragraphs of Rimes and IAM database yield results that are competitive with those of networks trained at line level, and constitute a significant step towards end-to-end transcription of full documents.

References in corpus (5)

• DRAW: A Recurrent Neural Network For Image Generation
• Multiple Object Recognition with Visual Attention
• End-to-end Continuous Speech Recognition using Attention-based Recurrent NN: First Results
• Recursive Recurrent Nets with Attention Modeling for OCR in the Wild
• Recurrent Spatial Transformer Networks

Cited by in corpus (8)

• Image Super-Resolution Using Very Deep Residual Channel Attention Networks
• OrigamiNet: Weakly-Supervised, Segmentation-Free, One-Step, Full Page Text Recognition by learning to unfold
• Teaching Machines to Code: Neural Markup Generation with Visual Attention
• TextNet: Irregular Text Reading from Images with an End-to-End Trainable Network
• Learning Spatial-Semantic Context with Fully Convolutional Recurrent Network for Online Handwritten Chinese Text Recognition
• Inductive Visual Localisation: Factorised Training for Superior Generalisation
• A Novel Integrated Framework for Learning both Text Detection and Recognition
• Robust End-to-End Offline Chinese Handwriting Text Page Spotter with Text Kernel