Subword Language Model for Query Auto-Completion

Abstract

Current neural query auto-completion (QAC) systems rely on character-level language models, but they slow down when queries are long. We present how to utilize subword language models for the fast and accurate generation of query completion candidates. Representing queries with subwords shorten a decoding length significantly. To deal with issues coming from introducing subword language model, we develop a retrace algorithm and a reranking method by approximate marginalization. As a result, our model achieves up to 2.5 times faster while maintaining a similar quality of generated results compared to the character-level baseline. Also, we propose a new evaluation metric, mean recoverable length (MRL), measuring how many upcoming characters the model could complete correctly. It provides more explicit meaning and eliminates the need for prefix length sampling for existing rank-based metrics. Moreover, we performed a comprehensive analysis with ablation study to figure out the importance of each component.

arXiv commentEMNLP-IJCNLP 2019; Code is available at https://github.com/clovaai/subword-qac ; 11 pages, 2 figures

References in corpus (20)

• Adam: A Method for Stochastic Optimization
• Distilling the Knowledge in a Neural Network
• Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
• Convolutional Neural Networks for Sentence Classification
• Character-Aware Neural Language Models
• A disciplined approach to neural network hyper-parameters: Part 1 -- learning rate, batch size, momentum, and weight decay
• Non-Autoregressive Neural Machine Translation
• Adaptive Computation Time for Recurrent Neural Networks
• Tying Word Vectors and Word Classifiers: A Loss Framework for Language Modeling
• Sequence-Level Knowledge Distillation
• NSML: Meet the MLaaS platform with a real-world case study
• Latent Predictor Networks for Code Generation
• Recurrent Dropout without Memory Loss
• NSML: A Machine Learning Platform That Enables You to Focus on Your Models
• Gated Word-Character Recurrent Language Model
• Sequence Modeling via Segmentations
• Task-Oriented Query Reformulation with Reinforcement Learning
• Personalized Language Model for Query Auto-Completion
• Personalized neural language models for real-world query auto completion
• Distilling Knowledge for Search-based Structured Prediction