Leveraging Adversarial Training in Self-Learning for Cross-Lingual Text Classification

Abstract

In cross-lingual text classification, one seeks to exploit labeled data from one language to train a text classification model that can then be applied to a completely different language. Recent multilingual representation models have made it much easier to achieve this. Still, there may still be subtle differences between languages that are neglected when doing so. To address this, we present a semi-supervised adversarial training process that minimizes the maximal loss for label-preserving input perturbations. The resulting model then serves as a teacher to induce labels for unlabeled target language samples that can be used during further adversarial training, allowing us to gradually adapt our model to the target language. Compared with a number of strong baselines, we observe significant gains in effectiveness on document and intent classification for a diverse set of languages.

SIGIR 2020 (Short Paper)

References in corpus (9)

• Cross-lingual Language Model Pretraining
• Towards Deep Learning Models Resistant to Adversarial Attacks
• Adversarial Training Methods for Semi-Supervised Text Classification
• How multilingual is Multilingual BERT?
• Massively Multilingual Sentence Embeddings for Zero-Shot Cross-Lingual Transfer and Beyond
• HotFlip: White-Box Adversarial Examples for Text Classification
• Robust Neural Machine Translation with Doubly Adversarial Inputs
• Cross-Lingual Transfer Learning for Multilingual Task Oriented Dialog
• Adversarial Learning with Contextual Embeddings for Zero-resource Cross-lingual Classification and NER