Evaluating and Characterizing Human Rationales
arXiv:2010.04736
Abstract
Two main approaches for evaluating the quality of machine-generated rationales are: 1) using human rationales as a gold standard; and 2) automated metrics based on how rationales affect model behavior. An open question, however, is how human rationales fare with these automatic metrics. Analyzing a variety of datasets and models, we find that human rationales do not necessarily perform well on these metrics. To unpack this finding, we propose improved metrics to account for model-dependent baseline performance. We then propose two methods to further characterize rationale quality, one based on model retraining and one on using "fidelity curves" to reveal properties such as irrelevance and redundancy. Our work leads to actionable suggestions for evaluating and characterizing rationales.
arXiv comment14 pages, 15 figures, to appear in EMNLP 2020. Code is available at https://github.com/BoulderDS/evaluating-human-rationales
References in corpus (5)
- A Unified Approach to Interpreting Model Predictions
- Axiomatic Attribution for Deep Networks
- On Human Predictions with Explanations and Predictions of Machine Learning Models: A Case Study on Deception Detection
- e-SNLI: Natural Language Inference with Natural Language Explanations
- BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questions