2kenize: Tying Subword Sequences for Chinese Script Conversion
arXiv:2005.03375
Abstract
Simplified Chinese to Traditional Chinese character conversion is a common preprocessing step in Chinese NLP. Despite this, current approaches have poor performance because they do not take into account that a simplified Chinese character can correspond to multiple traditional characters. Here, we propose a model that can disambiguate between mappings and convert between the two scripts. The model is based on subword segmentation, two language models, as well as a method for mapping between subword sequences. We further construct benchmark datasets for topic classification and script conversion. Our proposed method outperforms previous Chinese Character conversion approaches by 6 points in accuracy. These results are further confirmed in a downstream application, where 2kenize is used to convert pretraining dataset for topic classification. An error analysis reveals that our method's particular strengths are in dealing with code-mixing and named entities.
Accepted to ACL 2020
References in corpus (10)
- PyTorch: An Imperative Style, High-Performance Deep Learning Library
- Character-level Convolutional Networks for Text Classification
- fastai: A Layered API for Deep Learning
- A disciplined approach to neural network hyper-parameters: Part 1 -- learning rate, batch size, momentum, and weight decay
- Quasi-Recurrent Neural Networks
- An Empirical Model of Large-Batch Training
- Single Headed Attention RNN: Stop Thinking With Your Head
- Why Comparing Single Performance Scores Does Not Allow to Draw Conclusions About Machine Learning Approaches
- Learning to Create and Reuse Words in Open-Vocabulary Neural Language Modeling
- Universal Language Model Fine-Tuning with Subword Tokenization for Polish