Expanding the Text Classification Toolbox with Cross-Lingual Embeddings
arXiv:1903.09878
Abstract
Most work in text classification and Natural Language Processing (NLP) focuses on English or a handful of other languages that have text corpora of hundreds of millions of words. This is creating a new version of the digital divide: the artificial intelligence (AI) divide. Transfer-based approaches, such as Cross-Lingual Text Classification (CLTC) - the task of categorizing texts written in different languages into a common taxonomy, are a promising solution to the emerging AI divide. Recent work on CLTC has focused on demonstrating the benefits of using bilingual word embeddings as features, relegating the CLTC problem to a mere benchmark based on a simple averaged perceptron. In this paper, we explore more extensively and systematically two flavors of the CLTC problem: news topic classification and textual churn intent detection (TCID) in social media. In particular, we test the hypothesis that embeddings with context are more effective, by multi-tasking the learning of multilingual word embeddings and text classification; we explore neural architectures for CLTC; and we move from bi- to multi-lingual word embeddings. For all architectures, types of word embeddings and datasets, we notice a consistent gain trend in favor of multilingual joint training, especially for low-resourced languages.
References in corpus (6)
- Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
- Natural Language Processing (almost) from Scratch
- Massively Multilingual Word Embeddings
- Offline bilingual word vectors, orthogonal transformations and the inverted softmax
- Cross-lingual Models of Word Embeddings: An Empirical Comparison
- Churn Intent Detection in Multilingual Chatbot Conversations and Social Media