Cross-lingual Distillation for Text Classification
arXiv:1705.02073
Abstract
Cross-lingual text classification(CLTC) is the task of classifying documents written in different languages into the same taxonomy of categories. This paper presents a novel approach to CLTC that builds on model distillation, which adapts and extends a framework originally proposed for model compression. Using soft probabilistic predictions for the documents in a label-rich language as the (induced) supervisory labels in a parallel corpus of documents, we train classifiers successfully for new languages in which labeled training data are not available. An adversarial feature adaptation technique is also applied during the model training to reduce distribution mismatch. We conducted experiments on two benchmark CLTC datasets, treating English as the source language and German, French, Japan and Chinese as the unlabeled target languages. The proposed approach had the advantageous or comparable performance of the other state-of-art methods.
arXiv commentAccepted at ACL 2017; Code available at https://github.com/xrc10/cross-distill
References in corpus (13)
- Efficient Estimation of Word Representations in Vector Space
- Distilling the Knowledge in a Neural Network
- Sequence to Sequence Learning with Neural Networks
- FitNets: Hints for Thin Deep Nets
- Do Deep Nets Really Need to be Deep?
- Convolutional Neural Networks for Sentence Classification
- Effective Use of Word Order for Text Categorization with Convolutional Neural Networks
- Sequence-Level Knowledge Distillation
- Adversarial Deep Averaging Networks for Cross-Lingual Sentiment Classification
- Cross Modal Distillation for Supervision Transfer
- Distilling Word Embeddings: An Encoding Approach
- An Empirical Study on Sentiment Classification of Chinese Review using Word Embedding
- Dependency Sensitive Convolutional Neural Networks for Modeling Sentences and Documents