Teach Me to Explain: A Review of Datasets for Explainable Natural Language Processing
arXiv:2102.12060
Abstract
Explainable NLP (ExNLP) has increasingly focused on collecting human-annotated textual explanations. These explanations are used downstream in three ways: as data augmentation to improve performance on a predictive task, as supervision to train models to produce explanations for their predictions, and as a ground-truth to evaluate model-generated explanations. In this review, we identify 65 datasets with three predominant classes of textual explanations (highlights, free-text, and structured), organize the literature on annotating each type, identify strengths and shortcomings of existing collection methodologies, and give recommendations for collecting ExNLP datasets in the future.
arXiv commentv3: NeurIPS 2021 accepted paper camera-ready version. The content of v3 is almost the same as of v1-2 but is more condensed. v4: Fixed a typo in the title and added acknowledgements. 10 pages main, 6 pages appendix
References in corpus (6)
- A Survey on the Explainability of Supervised Machine Learning
- Towards Robust Interpretability with Self-Explaining Neural Networks
- Performance Impact Caused by Hidden Bias of Training Data for Recognizing Textual Entailment
- Data and its (dis)contents: A survey of dataset development and use in machine learning research
- Polyjuice: Generating Counterfactuals for Explaining, Evaluating, and Improving Models
- Learning from Explanations with Neural Execution Tree