A Survey on Text Classification: From Shallow to Deep Learning

Abstract

Text classification is the most fundamental and essential task in natural language processing. The last decade has seen a surge of research in this area due to the unprecedented success of deep learning. Numerous methods, datasets, and evaluation metrics have been proposed in the literature, raising the need for a comprehensive and updated survey. This paper fills the gap by reviewing the state-of-the-art approaches from 1961 to 2021, focusing on models from traditional models to deep learning. We create a taxonomy for text classification according to the text involved and the models used for feature extraction and classification. We then discuss each of these categories in detail, dealing with both the technical developments and benchmark datasets that support tests of predictions. A comprehensive comparison between different techniques, as well as identifying the pros and cons of various evaluation metrics are also provided in this survey. Finally, we conclude by summarizing key implications, future research directions, and the challenges facing the research area.

References in corpus (35)

• Efficient Estimation of Word Representations in Vector Space
• XLNet: Generalized Autoregressive Pretraining for Language Understanding
• A Structured Self-attentive Sentence Embedding
• Deep Convolutional Networks on Graph-Structured Data
• Simplifying Graph Convolutional Networks
• Recurrent Neural Network for Text Classification with Multi-Task Learning
• SQuAD: 100,000+ Questions for Machine Comprehension of Text
• ERNIE: Enhanced Representation through Knowledge Integration
• Seeing stars: Exploiting class relationships for sentiment categorization with respect to rating scales
• A Convolutional Neural Network for Modelling Sentences
• HDLTex: Hierarchical Deep Learning for Text Classification
• A large annotated corpus for learning natural language inference
• Deep Learning Based Text Classification: A Comprehensive Review
• Distributional Smoothing with Virtual Adversarial Training
• Attentive Pooling Networks
• Text Classification Improved by Integrating Bidirectional LSTM with Two-dimensional Max Pooling
• Semi-supervised Convolutional Neural Networks for Text Categorization via Region Embedding
• Learning Natural Language Inference using Bidirectional LSTM model and Inner-Attention
• Multi-Task Deep Neural Networks for Natural Language Understanding
• Know What You Don't Know: Unanswerable Questions for SQuAD
• Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learning
• Long Short-Term Memory-Networks for Machine Reading
• TopicRNN: A Recurrent Neural Network with Long-Range Semantic Dependency
• Interactive Attention Networks for Aspect-Level Sentiment Classification
• Bilateral Multi-Perspective Matching for Natural Language Sentences
• Investigating Capsule Networks with Dynamic Routing for Text Classification
• Multi-Label Text Classification using Attention-based Graph Neural Network
• AttentionXML: Label Tree-based Attention-Aware Deep Model for High-Performance Extreme Multi-Label Text Classification
• Bi-Directional Block Self-Attention for Fast and Memory-Efficient Sequence Modeling
• Few-shot Text Classification with Distributional Signatures
• Every Document Owns Its Structure: Inductive Text Classification via Graph Neural Networks
• Constructing Narrative Event Evolutionary Graph for Script Event Prediction
• Scalable Twin Neural Networks for Classification of Unbalanced Data
• Text Level Graph Neural Network for Text Classification
• Event Representation Learning Enhanced with External Commonsense Knowledge

Cited by in corpus (7)

• Improving Short Text Classification With Augmented Data Using GPT-3
• Detecting Privacy Requirements from User Stories with NLP Transfer Learning Models
• Embedding generation for text classification of Brazilian Portuguese user reviews: from bag-of-words to transformers
• Model Extraction and Adversarial Transferability, Your BERT is Vulnerable!
• HTCInfoMax: A Global Model for Hierarchical Text Classification via Information Maximization
• Incremental Few-shot Text Classification with Multi-round New Classes: Formulation, Dataset and System
• Hierarchical Heterogeneous Graph Representation Learning for Short Text Classification