A Multilayer Convolutional Encoder-Decoder Neural Network for Grammatical Error Correction

Abstract

We improve automatic correction of grammatical, orthographic, and collocation errors in text using a multilayer convolutional encoder-decoder neural network. The network is initialized with embeddings that make use of character N-gram information to better suit this task. When evaluated on common benchmark test data sets (CoNLL-2014 and JFLEG), our model substantially outperforms all prior neural approaches on this task as well as strong statistical machine translation-based systems with neural and task-specific features trained on the same data. Our analysis shows the superiority of convolutional neural networks over recurrent neural networks such as long short-term memory (LSTM) networks in capturing the local context via attention, and thereby improving the coverage in correcting grammatical errors. By ensembling multiple models, and incorporating an N-gram language model and edit features via rescoring, our novel method becomes the first neural approach to outperform the current state-of-the-art statistical machine translation-based approach, both in terms of grammaticality and fluency.

8 pages, 3 figures, In Proceedings of AAAI 2018

Cited by in corpus (16)

• Tree-Transformer: A Transformer-Based Method for Correction of Tree-Structured Data
• A Comprehensive Survey of Grammar Error Correction
• Spelling Error Correction Using a Nested RNN Model and Pseudo Training Data
• Sequence-to-sequence Pre-training with Data Augmentation for Sentence Rewriting
• Understanding and Improving Encoder Layer Fusion in Sequence-to-Sequence Learning
• ENCORE: Ensemble Learning using Convolution Neural Machine Translation for Automatic Program Repair
• An Empirical Study of Incorporating Pseudo Data into Grammatical Error Correction
• Synthetic Data Generation for Grammatical Error Correction with Tagged Corruption Models
• Parallel Iterative Edit Models for Local Sequence Transduction
• Recent Trends in the Use of Deep Learning Models for Grammar Error Handling
• A Syntax-Guided Grammatical Error Correction Model with Dependency Tree Correction
• Chinese Grammatical Correction Using BERT-based Pre-trained Model
• LM-Critic: Language Models for Unsupervised Grammatical Error Correction
• Few-Shot Domain Adaptation for Grammatical Error Correction via Meta-Learning
• Comparison of Grammatical Error Correction Using Back-Translation Models
• A Self-Refinement Strategy for Noise Reduction in Grammatical Error Correction