Adversarial Training for Large Neural Language Models

Abstract

Generalization and robustness are both key desiderata for designing machine learning methods. Adversarial training can enhance robustness, but past work often finds it hurts generalization. In natural language processing (NLP), pre-training large neural language models such as BERT have demonstrated impressive gain in generalization for a variety of tasks, with further improvement from adversarial fine-tuning. However, these models are still vulnerable to adversarial attacks. In this paper, we show that adversarial pre-training can improve both generalization and robustness. We propose a general algorithm ALUM (Adversarial training for large neural LangUage Models), which regularizes the training objective by applying perturbations in the embedding space that maximizes the adversarial loss. We present the first comprehensive study of adversarial training in all stages, including pre-training from scratch, continual pre-training on a well-trained model, and task-specific fine-tuning. ALUM obtains substantial gains over BERT on a wide range of NLP tasks, in both regular and adversarial scenarios. Even for models that have been well trained on extremely large text corpora, such as RoBERTa, ALUM can still produce significant gains from continual pre-training, whereas conventional non-adversarial methods can not. ALUM can be further combined with task-specific fine-tuning to attain additional gains. The ALUM code is publicly available at https://github.com/namisan/mt-dnn.

13 pages, 9 tables, 2 figures

References in corpus (17)

• Adam: A Method for Stochastic Optimization
• Unified Language Model Pre-training for Natural Language Understanding and Generation
• Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism
• Deep Learning Based Text Classification: A Comprehensive Review
• A Simple Method for Commonsense Reasoning
• FreeLB: Enhanced Adversarial Training for Natural Language Understanding
• Adversarial Examples for Evaluating Reading Comprehension Systems
• UniLMv2: Pseudo-Masked Language Models for Unified Language Model Pre-Training
• Adversarial Robustness through Local Linearization
• Adversarial Training Can Hurt Generalization
• Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailment
• ERNIE 2.0: A Continual Pre-training Framework for Language Understanding
• Adversarial NLI: A New Benchmark for Natural Language Understanding
• Improving Neural Language Modeling via Adversarial Training
• Understanding and Mitigating the Tradeoff Between Robustness and Accuracy
• The Curious Case of Adversarially Robust Models: More Data Can Help, Double Descend, or Hurt Generalization
• The Microsoft Toolkit of Multi-Task Deep Neural Networks for Natural Language Understanding

Cited by in corpus (37)

• Language Models are Few-Shot Learners
• Domain-Specific Language Model Pretraining for Biomedical Natural Language Processing
• Pre-trained Models for Natural Language Processing: A Survey
• A Survey on Large Language Model (LLM) Security and Privacy: The Good, the Bad, and the Ugly
• DeBERTa: Decoding-enhanced BERT with Disentangled Attention
• A Survey on Data Augmentation for Text Classification
• Deep Learning Based Text Classification: A Comprehensive Review
• Large-Scale Adversarial Training for Vision-and-Language Representation Learning
• A Simple but Tough-to-Beat Data Augmentation Approach for Natural Language Understanding and Generation
• Very Deep Transformers for Neural Machine Translation
• InfoBERT: Improving Robustness of Language Models from An Information Theoretic Perspective
• CoDA: Contrast-enhanced and Diversity-promoting Data Augmentation for Natural Language Understanding
• A Closer Look at the Robustness of Vision-and-Language Pre-trained Models
• Neural Language Generation: Formulation, Methods, and Evaluation
• Adversarial Reinforced Instruction Attacker for Robust Vision-Language Navigation
• Self-supervised Pre-training with Hard Examples Improves Visual Representations
• The Microsoft Toolkit of Multi-Task Deep Neural Networks for Natural Language Understanding
• Generating Plausible Counterfactual Explanations for Deep Transformers in Financial Text Classification
• Adversarial Training for Machine Reading Comprehension with Virtual Embeddings
• RADDLE: An Evaluation Benchmark and Analysis Platform for Robust Task-oriented Dialog Systems
• Simple Contrastive Representation Adversarial Learning for NLP Tasks
• Towards Improving Adversarial Training of NLP Models
• HiddenCut: Simple Data Augmentation for Natural Language Understanding with Better Generalization
• Consistency Regularization for Cross-Lingual Fine-Tuning
• How Does Adversarial Fine-Tuning Benefit BERT?
• Sequential Attention Module for Natural Language Processing
• Enhancing Transformers with Gradient Boosted Decision Trees for NLI Fine-Tuning
• Targeted Adversarial Training for Natural Language Understanding
• OCHADAI-KYOTO at SemEval-2021 Task 1: Enhancing Model Generalization and Robustness for Lexical Complexity Prediction
• Posterior Differential Regularization with f-divergence for Improving Model Robustness
• Effective Unsupervised Domain Adaptation with Adversarially Trained Language Models
• ARCH: Efficient Adversarial Regularized Training with Caching
• Adversarial Regularization as Stackelberg Game: An Unrolled Optimization Approach
• MagicPai at SemEval-2021 Task 7: Method for Detecting and Rating Humor Based on Multi-Task Adversarial Training
• MvSR-NAT: Multi-view Subset Regularization for Non-Autoregressive Machine Translation
• Adversarial Examples Generation for Reducing Implicit Gender Bias in Pre-trained Models
• Muppet: Massive Multi-task Representations with Pre-Finetuning