A GRU-based Encoder-Decoder Approach with Attention for Online Handwritten Mathematical Expression Recognition
arXiv:1712.03991
Abstract
In this study, we present a novel end-to-end approach based on the encoder-decoder framework with the attention mechanism for online handwritten mathematical expression recognition (OHMER). First, the input two-dimensional ink trajectory information of handwritten expression is encoded via the gated recurrent unit based recurrent neural network (GRU-RNN). Then the decoder is also implemented by the GRU-RNN with a coverage-based attention model. The proposed approach can simultaneously accomplish the symbol recognition and structural analysis to output a character sequence in LaTeX format. Validated on the CROHME 2014 competition task, our approach significantly outperforms the state-of-the-art with an expression recognition accuracy of 52.43% by only using the official training dataset. Furthermore, the alignments between the input trajectories of handwritten expressions and the output LaTeX sequences are visualized by the attention mechanism to show the effectiveness of the proposed method.
Accepted by ICDAR 2017 conference
References in corpus (6)
- Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
- ADADELTA: An Adaptive Learning Rate Method
- End-to-end Continuous Speech Recognition using Attention-based Recurrent NN: First Results
- Modeling Coverage for Neural Machine Translation
- Natural Language Understanding with Distributed Representation
- Drawing and Recognizing Chinese Characters with Recurrent Neural Network
Cited by in corpus (4)
- Sliding Line Point Regression for Shape Robust Scene Text Detection
- Multi-Scale Attention with Dense Encoder for Handwritten Mathematical Expression Recognition
- Trajectory-based Radical Analysis Network for Online Handwritten Chinese Character Recognition
- Radical analysis network for zero-shot learning in printed Chinese character recognition