Towards Interpreting and Mitigating Shortcut Learning Behavior of NLU Models
arXiv:2103.06922
Abstract
Recent studies indicate that NLU models are prone to rely on shortcut features for prediction, without achieving true language understanding. As a result, these models fail to generalize to real-world out-of-distribution data. In this work, we show that the words in the NLU training set can be modeled as a long-tailed distribution. There are two findings: 1) NLU models have strong preference for features located at the head of the long-tailed distribution, and 2) Shortcut features are picked up during very early few iterations of the model training. These two observations are further employed to formulate a measurement which can quantify the shortcut degree of each training sample. Based on this shortcut measurement, we propose a shortcut mitigation framework LTGR, to suppress the model from making overconfident predictions for samples with large shortcut degree. Experimental results on three NLU benchmarks demonstrate that our long-tailed distribution explanation accurately reflects the shortcut learning behavior of NLU models. Experimental analysis further indicates that LTGR can improve the generalization accuracy on OOD data, while preserving the accuracy on in-distribution data.
Accepted by NAACL 2021
References in corpus (15)
- Distilling the Knowledge in a Neural Network
- Axiomatic Attribution for Deep Networks
- Methods for Interpreting and Understanding Deep Neural Networks
- Shortcut Learning in Deep Neural Networks
- When Does Label Smoothing Help?
- Embedding Watermarks into Deep Neural Networks
- Neural Natural Language Inference Models Enhanced with External Knowledge
- Measuring the tendency of CNNs to Learn Surface Statistical Regularities
- Learning De-biased Representations with Biased Representations
- Learning Robust Representations by Projecting Superficial Statistics Out
- The Pitfalls of Simplicity Bias in Neural Networks
- Weight Poisoning Attacks on Pre-trained Models
- What does BERT Learn from Multiple-Choice Reading Comprehension Datasets?
- Deep Serial Number: Computational Watermarking for DNN Intellectual Property Protection
- A Unified Taylor Framework for Revisiting Attribution Methods