A psychophysics approach for quantitative comparison of interpretable computer vision models

Abstract

The field of transparent Machine Learning (ML) has contributed many novel methods aiming at better interpretability for computer vision and ML models in general. But how useful the explanations provided by transparent ML methods are for humans remains difficult to assess. Most studies evaluate interpretability in qualitative comparisons, they use experimental paradigms that do not allow for direct comparisons amongst methods or they report only offline experiments with no humans in the loop. While there are clear advantages of evaluations with no humans in the loop, such as scalability, reproducibility and less algorithmic bias than with humans in the loop, these metrics are limited in their usefulness if we do not understand how they relate to other metrics that take human cognition into account. Here we investigate the quality of interpretable computer vision algorithms using techniques from psychophysics. In crowdsourced annotation tasks we study the impact of different interpretability approaches on annotation accuracy and task time. In order to relate these findings to quality measures for interpretability without humans in the loop we compare quality metrics with and without humans in the loop. Our results demonstrate that psychophysical experiments allow for robust quality assessment of transparency in machine learning. Interestingly the quality metrics computed without humans in the loop did not provide a consistent ranking of interpretability methods nor were they representative for how useful an explanation was for humans. These findings highlight the potential of methods from classical psychophysics for modern machine learning applications. We hope that our results provide convincing arguments for evaluating interpretability in its natural habitat, human-ML interaction, if the goal is to obtain an authentic assessment of interpretability.

References in corpus (9)

• Towards A Rigorous Science of Interpretable Machine Learning
• Striving for Simplicity: The All Convolutional Net
• Explaining NonLinear Classification Decisions with Deep Taylor Decomposition
• Understanding Black-box Predictions via Influence Functions
• Quantifying Interpretability and Trust in Machine Learning Systems
• The Doctor Just Won't Accept That!
• The Promise and Peril of Human Evaluation for Model Interpretability
• Understanding and Comparing Deep Neural Networks for Age and Gender Classification
• iNNvestigate neural networks!

Cited by in corpus (3)

• How explainable AI affects human performance: A systematic review of the behavioural consequences of saliency maps
• How Well do Feature Visualizations Support Causal Understanding of CNN Activations?
• Exemplary Natural Images Explain CNN Activations Better than State-of-the-Art Feature Visualization