Human-in-the-Loop Interpretability Prior

Abstract

We often desire our models to be interpretable as well as accurate. Prior work on optimizing models for interpretability has relied on easy-to-quantify proxies for interpretability, such as sparsity or the number of operations required. In this work, we optimize for interpretability by directly including humans in the optimization loop. We develop an algorithm that minimizes the number of user studies to find models that are both predictive and interpretable and demonstrate our approach on several data sets. Our human subjects results show trends towards different proxy notions of interpretability on different datasets, which suggests that different proxies are preferred on different tasks.

arXiv commentTo appear at NIPS 2018, selected for a spotlight. 13 pages (incl references and appendix)

Cited by in corpus (9)

• Human Factors in Model Interpretability: Industry Practices, Challenges, and Needs
• Conceptual Metaphors Impact Perceptions of Human-AI Collaboration
• The TA Framework: Designing Real-time Teaching Augmentation for K-12 Classrooms
• Interactive Dimensionality Reduction for Comparative Analysis
• Assessing the Local Interpretability of Machine Learning Models
• Artificial Intelligence as Structural Estimation: Economic Interpretations of Deep Blue, Bonanza, and AlphaGo
• Cost-effective Interactive Attention Learning with Neural Attention Processes
• Good-looking but Lacking Faithfulness: Understanding Local Explanation Methods through Trend-based Testing
• Robust Semantic Interpretability: Revisiting Concept Activation Vectors