On Data Augmentation for Extreme Multi-label Classification

Abstract

In this paper, we focus on data augmentation for the extreme multi-label classification (XMC) problem. One of the most challenging issues of XMC is the long tail label distribution where even strong models suffer from insufficient supervision. To mitigate such label bias, we propose a simple and effective augmentation framework and a new state-of-the-art classifier. Our augmentation framework takes advantage of the pre-trained GPT-2 model to generate label-invariant perturbations of the input texts to augment the existing training data. As a result, it present substantial improvements over baseline models. Our contributions are two-factored: (1) we introduce a new state-of-the-art classifier that uses label attention with RoBERTa and combine it with our augmentation framework for further improvement; (2) we present a broad study on how effective are different augmentation methods in the XMC task.

References in corpus (8)

• Unsupervised Data Augmentation for Consistency Training
• Data Augmentation using Pre-trained Transformer Models
• LSHTC: A Benchmark for Large-Scale Text Classification
• AttentionXML: Label Tree-based Attention-Aware Deep Model for High-Performance Extreme Multi-Label Text Classification
• Enhancing Navigation on Wikipedia with Social Tags
• DARE: Data Augmented Relation Extraction with GPT-2
• Taming Pretrained Transformers for Extreme Multi-label Text Classification
• Adversarial Extreme Multi-label Classification

Cited by in corpus (4)

• Data Augmentation Approaches in Natural Language Processing: A Survey
• Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Survey
• Textual Data Augmentation for Patient Outcomes Prediction
• QUEACO: Borrowing Treasures from Weakly-labeled Behavior Data for Query Attribute Value Extraction