Multilingual and Multilabel Emotion Recognition using Virtual Adversarial Training

Abstract

Virtual Adversarial Training (VAT) has been effective in learning robust models under supervised and semi-supervised settings for both computer vision and NLP tasks. However, the efficacy of VAT for multilingual and multilabel text classification has not been explored before. In this work, we explore VAT for multilabel emotion recognition with a focus on leveraging unlabelled data from different languages to improve the model performance. We perform extensive semi-supervised experiments on SemEval2018 multilabel and multilingual emotion recognition dataset and show performance gains of 6.2% (Arabic), 3.8% (Spanish) and 1.8% (English) over supervised learning with same amount of labelled data (10% of training data). We also improve the existing state-of-the-art by 7%, 4.5% and 1% (Jaccard Index) for Spanish, Arabic and English respectively and perform probing experiments for understanding the impact of different layers of the contextual models.

References in corpus (6)

• BioBERT: a pre-trained biomedical language representation model for biomedical text mining
• Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results
• AraBERT: Transformer-based Model for Arabic Language Understanding
• FreeLB: Enhanced Adversarial Training for Natural Language Understanding
• Characterizing Adversarial Examples Based on Spatial Consistency Information for Semantic Segmentation
• Seq2Emo for Multi-label Emotion Classification Based on Latent Variable Chains Transformation