Spelling Correction with Denoising Transformer
arXiv:2105.05977
Abstract
We present a novel method of performing spelling correction on short input strings, such as search queries or individual words. At its core lies a procedure for generating artificial typos which closely follow the error patterns manifested by humans. This procedure is used to train the production spelling correction model based on a transformer architecture. This model is currently served in the HubSpot product search. We show that our approach to typo generation is superior to the widespread practice of adding noise, which ignores human patterns. We also demonstrate how our approach may be extended to resource-scarce settings and train spelling correction models for Arabic, Greek, Russian, and Setswana languages, without using any labeled data.
arXiv comment9 pages, 3 figures
References in corpus (7)
- MaskGAN: Better Text Generation via Filling in the______
- Neural Networks for Text Correction and Completion in Keyboard Decoding
- Gmail Smart Compose: Real-Time Assisted Writing
- A Focus on Neural Machine Translation for African Languages
- Parallel Spell-Checking Algorithm Based on Yahoo! N-Grams Dataset
- Spelling Correction as a Foreign Language
- Denoising based Sequence-to-Sequence Pre-training for Text Generation