Interpretable and Steerable Sequence Learning via Prototypes

Abstract

One of the major challenges in machine learning nowadays is to provide predictions with not only high accuracy but also user-friendly explanations. Although in recent years we have witnessed increasingly popular use of deep neural networks for sequence modeling, it is still challenging to explain the rationales behind the model outputs, which is essential for building trust and supporting the domain experts to validate, critique and refine the model. We propose ProSeNet, an interpretable and steerable deep sequence model with natural explanations derived from case-based reasoning. The prediction is obtained by comparing the inputs to a few prototypes, which are exemplar cases in the problem domain. For better interpretability, we define several criteria for constructing the prototypes, including simplicity, diversity, and sparsity and propose the learning objective and the optimization procedure. ProSeNet also provides a user-friendly approach to model steering: domain experts without any knowledge on the underlying model or parameters can easily incorporate their intuition and experience by manually refining the prototypes. We conduct experiments on a wide range of real-world applications, including predictive diagnostics for automobiles, ECG, and protein sequence classification and sentiment analysis on texts. The result shows that ProSeNet can achieve accuracy on par with state-of-the-art deep learning models. We also evaluate the interpretability of the results with concrete case studies. Finally, through user study on Amazon Mechanical Turk (MTurk), we demonstrate that the model selects high-quality prototypes which align well with human knowledge and can be interactively refined for better interpretability without loss of performance.

Accepted as a full paper at KDD 2019 on May 8, 2019

References in corpus (1)

• Automatic Text Scoring Using Neural Networks

Cited by in corpus (11)

• The State of the Art in Enhancing Trust in Machine Learning Models with the Use of Visualizations
• A Survey on ML4VIS: Applying Machine Learning Advances to Data Visualization
• ProtoPShare: Prototype Sharing for Interpretable Image Classification and Similarity Discovery
• This Looks Like That, Because ... Explaining Prototypes for Interpretable Image Recognition
• RASIPAM: Interactive Pattern Mining of Multivariate Event Sequences in Racket Sports
• A Comprehensive Survey on Self-Interpretable Neural Networks
• Unveiling Global Interactive Patterns across Graphs: Towards Interpretable Graph Neural Networks
• This Reads Like That: Deep Learning for Interpretable Natural Language Processing
• Multi-Scale Grouped Prototypes for Interpretable Semantic Segmentation
• Comprehensive Evaluation of Prototype Neural Networks
• ProtoTSNet: Interpretable Multivariate Time Series Classification With Prototypical Parts