Understanding Convolutional Neural Networks for Text Classification

Abstract

We present an analysis into the inner workings of Convolutional Neural Networks (CNNs) for processing text. CNNs used for computer vision can be interpreted by projecting filters into image space, but for discrete sequence inputs CNNs remain a mystery. We aim to understand the method by which the networks process and classify text. We examine common hypotheses to this problem: that filters, accompanied by global max-pooling, serve as ngram detectors. We show that filters may capture several different semantic classes of ngrams by using different activation patterns, and that global max-pooling induces behavior which separates important ngrams from the rest. Finally, we show practical use cases derived from our findings in the form of model interpretability (explaining a trained model by deriving a concrete identity for each filter, bridging the gap between visualization tools in vision tasks and NLP) and prediction interpretability (explaining predictions). Code implementation is available online at github.com/sayaendo/interpreting-cnn-for-text.

arXiv commentAccepted to "Analyzing and interpreting neural networks for NLP" workshop in EMNLP 2018. v2: Added link to online github implementation

References in corpus (3)

• Natural Language Processing (almost) from Scratch
• An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling
• Seeing stars: Exploiting class relationships for sentiment categorization with respect to rating scales

Cited by in corpus (10)

• SHAP values for Explaining CNN-based Text Classification Models
• Structural block driven - enhanced convolutional neural representation for relation extraction
• Model Explainability in Deep Learning Based Natural Language Processing
• ProtoryNet - Interpretable Text Classification Via Prototype Trajectories
• ShufText: A Simple Black Box Approach to Evaluate the Fragility of Text Classification Models
• Interpretable Text Classification Using CNN and Max-pooling
• Self-interpretable Convolutional Neural Networks for Text Classification
• Understanding Information Spreading Mechanisms During COVID-19 Pandemic by Analyzing the Impact of Tweet Text and User Features for Retweet Prediction
• TechTexC: Classification of Technical Texts using Convolution and Bidirectional Long Short Term Memory Network
• Unsupervised Technique To Conversational Machine Reading