Targeted Adversarial Training for Natural Language Understanding
arXiv:2104.05847
Abstract
We present a simple yet effective Targeted Adversarial Training (TAT) algorithm to improve adversarial training for natural language understanding. The key idea is to introspect current mistakes and prioritize adversarial training steps to where the model errs the most. Experiments show that TAT can significantly improve accuracy over standard adversarial training on GLUE and attain new state-of-the-art zero-shot results on XNLI. Our code will be released at: https://github.com/namisan/mt-dnn.
arXiv comment9 pages, 4 tables, 3 figurers, NAACL 2021
References in corpus (6)
- Adam: A Method for Stochastic Optimization
- Lookahead Optimizer: k steps forward, 1 step back
- Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailment
- Adversarial Training for Large Neural Language Models
- InfoXLM: An Information-Theoretic Framework for Cross-Lingual Language Model Pre-Training
- Adversarial Training for Commonsense Inference