Open Vocabulary Learning for Neural Chinese Pinyin IME

Abstract

Pinyin-to-character (P2C) conversion is the core component of pinyin-based Chinese input method engine (IME). However, the conversion is seriously compromised by the ambiguities of Chinese characters corresponding to pinyin as well as the predefined fixed vocabularies. To alleviate such inconveniences, we propose a neural P2C conversion model augmented by an online updated vocabulary with a sampling mechanism to support open vocabulary learning during IME working. Our experiments show that the proposed method outperforms commercial IMEs and state-of-the-art traditional models on standard corpus and true inputting history dataset in terms of multiple metrics and thus the online updated vocabulary indeed helps our IME effectively follows user inputting behavior.

arXiv commentAccepted by ACL 2019

References in corpus (9)

• Efficient Estimation of Word Representations in Vector Space
• Modeling Multi-turn Conversation with Deep Utterance Aggregation
• Syntax-Directed Attention for Neural Machine Translation
• Neural Response Generation with Dynamic Vocabularies
• Lingke: A Fine-grained Multi-turn Chatbot for Customer Service
• Vocabulary Selection Strategies for Neural Machine Translation
• One-shot Learning for Question-Answering in Gaokao History Challenge
• Dependency or Span, End-to-End Uniform Semantic Role Labeling
• Tracing a Loose Wordhood for Chinese Input Method Engine