ShufText: A Simple Black Box Approach to Evaluate the Fragility of Text Classification Models

Abstract

Text classification is the most basic natural language processing task. It has a wide range of applications ranging from sentiment analysis to topic classification. Recently, deep learning approaches based on CNN, LSTM, and Transformers have been the de facto approach for text classification. In this work, we highlight a common issue associated with these approaches. We show that these systems are over-reliant on the important words present in the text that are useful for classification. With limited training data and discriminative training strategy, these approaches tend to ignore the semantic meaning of the sentence and rather just focus on keywords or important n-grams. We propose a simple black box technique ShutText to present the shortcomings of the model and identify the over-reliance of the model on keywords. This involves randomly shuffling the words in a sentence and evaluating the classification accuracy. We see that on common text classification datasets there is very little effect of shuffling and with high probability these models predict the original class. We also evaluate the effect of language model pretraining on these models and try to answer questions around model robustness to out of domain sentences. We show that simple models based on CNN or LSTM as well as complex models like BERT are questionable in terms of their syntactic and semantic understanding.

References in corpus (11)

• Text Classification Algorithms: A Survey
• A C-LSTM Neural Network for Text Classification
• Understanding Neural Networks through Representation Erasure
• Deep Text Classification Can be Fooled
• What you can cram into a single vector: Probing sentence embeddings for linguistic properties
• Pathologies of Neural Models Make Interpretations Difficult
• Deep Learning for Hindi Text Classification: A Comparison
• Experimental Evaluation of Deep Learning models for Marathi Text Classification
• Understanding Convolutional Neural Networks for Text Classification
• AllenNLP Interpret: A Framework for Explaining Predictions of NLP Models
• Benchmarking Popular Classification Models' Robustness to Random and Targeted Corruptions

Cited by in corpus (2)

• Automatic Construction of Evaluation Suites for Natural Language Generation Datasets
• On Sensitivity of Deep Learning Based Text Classification Algorithms to Practical Input Perturbations