Deep Learning Based Text Classification: A Comprehensive Review

Abstract

Deep learning based models have surpassed classical machine learning based approaches in various text classification tasks, including sentiment analysis, news categorization, question answering, and natural language inference. In this paper, we provide a comprehensive review of more than 150 deep learning based models for text classification developed in recent years, and discuss their technical contributions, similarities, and strengths. We also provide a summary of more than 40 popular datasets widely used for text classification. Finally, we provide a quantitative analysis of the performance of different deep learning models on popular benchmarks, and discuss future research directions.

References in corpus (64)

• Efficient Estimation of Word Representations in Vector Space
• Auto-Encoding Variational Bayes
• Semi-Supervised Classification with Graph Convolutional Networks
• Natural Language Processing (almost) from Scratch
• Distributed Representations of Sentences and Documents
• Language Models are Few-Shot Learners
• Convolutional Sequence to Sequence Learning
• Domain-Specific Language Model Pretraining for Biomedical Natural Language Processing
• Unsupervised Data Augmentation for Consistency Training
• Pre-trained Models for Natural Language Processing: A Survey
• A Structured Self-attentive Sentence Embedding
• Training Very Deep Networks
• Simplifying Graph Convolutional Networks
• Recurrent Neural Network for Text Classification with Multi-Task Learning
• Character-Aware Neural Language Models
• SQuAD: 100,000+ Questions for Machine Comprehension of Text
• ERNIE: Enhanced Representation through Knowledge Integration
• Skip-Thought Vectors
• Dynamic Memory Networks for Visual and Textual Question Answering
• End-To-End Memory Networks
• A C-LSTM Neural Network for Text Classification
• MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers
• Semi-supervised Sequence Learning
• HDLTex: Hierarchical Deep Learning for Text Classification
• A large annotated corpus for learning natural language inference
• LSTM-based Deep Learning Models for Non-factoid Answer Selection
• GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding
• Distributional Smoothing with Virtual Adversarial Training
• Distilling Task-Specific Knowledge from BERT into Simple Neural Networks
• Attentive Pooling Networks
• Text Classification Improved by Integrating Bidirectional LSTM with Two-dimensional Max Pooling
• The Next Decade in AI: Four Steps Towards Robust Artificial Intelligence
• ReasoNet: Learning to Stop Reading in Machine Comprehension
• Learning Natural Language Inference using Bidirectional LSTM model and Inner-Attention
• UniLMv2: Pseudo-Masked Language Models for Unified Language Model Pre-Training
• Multi-Task Deep Neural Networks for Natural Language Understanding
• Adversarial Training Methods for Semi-Supervised Text Classification
• Medical Text Classification using Convolutional Neural Networks
• Long Short-Term Memory-Networks for Machine Reading
• Weakly-Supervised Neural Text Classification
• A Deep Architecture for Semantic Matching with Multiple Positional Sentence Representations
• Efficient Character-level Document Classification by Combining Convolution and Recurrent Layers
• FusionNet: Fusing via Fully-Aware Attention with Application to Machine Comprehension
• TopicRNN: A Recurrent Neural Network with Long-Range Semantic Dependency
• Supervised and Semi-Supervised Text Categorization using LSTM for Region Embeddings
• Investigating Capsule Networks with Dynamic Routing for Text Classification
• Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailment
• CondenseNet: An Efficient DenseNet using Learned Group Convolutions
• Adversarial Multi-task Learning for Text Classification
• Adversarial Training for Large Neural Language Models
• Adversarial NLI: A New Benchmark for Natural Language Understanding
• Text Classification using Capsules
• A Decomposable Attention Model for Natural Language Inference
• Learning text representation using recurrent convolutional neural network with highway layers
• Joint Embedding of Words and Labels for Text Classification
• Enhancing the Transformer with Explicit Relational Encoding for Math Problem Solving
• Stochastic Answer Networks for Machine Reading Comprehension
• TwinBERT: Distilling Knowledge to Twin-Structured BERT Models for Efficient Retrieval
• Reinforced Self-Attention Network: a Hybrid of Hard and Soft Attention for Sequence Modeling
• Recurrent Neural Network-Based Sentence Encoder with Gated Attention for Natural Language Inference
• Robust Conversational AI with Grounded Text Generation
• Modelling Interaction of Sentence Pair with coupled-LSTMs
• Text Level Graph Neural Network for Text Classification
• Neural Attentive Bag-of-Entities Model for Text Classification

Cited by in corpus (35)

• DeBERTa: Decoding-enhanced BERT with Disentangled Attention
• A Benchmark Study of Machine Learning Models for Online Fake News Detection
• Natural Language Processing Advancements By Deep Learning: A Survey
• Review: Deep Learning in Electron Microscopy
• A Survey on Text Classification: From Shallow to Deep Learning
• Adversarial Training for Large Neural Language Models
• An Online Multilingual Hate speech Recognition System
• Very Deep Transformers for Neural Machine Translation
• A Comprehensive Survey of Text Classification Techniques and Their Research Applications: Observational and Experimental Insights
• Natural Language Understanding for Argumentative Dialogue Systems in the Opinion Building Domain
• BERTERS: Multimodal Representation Learning for Expert Recommendation System with Transformer
• Towards Transmission-Friendly and Robust CNN Models over Cloud and Device
• HeteGCN: Heterogeneous Graph Convolutional Networks for Text Classification
• Covid-Transformer: Detecting COVID-19 Trending Topics on Twitter Using Universal Sentence Encoder
• Molecular Identification from AFM images using the IUPAC Nomenclature and Attribute Multimodal Recurrent Neural Networks
• Generalised gravitational burst generation with Generative Adversarial Networks
• Multimodal price prediction
• A Model to Measure the Spread Power of Rumors
• Embedding generation for text classification of Brazilian Portuguese user reviews: from bag-of-words to transformers
• Entity Linking Meets Deep Learning: Techniques and Solutions
• Automatic Analysis of Available Source Code of Top Artificial Intelligence Conference Papers
• Extending Logic Explained Networks to Text Classification
• BERT based classification system for detecting rumours on Twitter
• Classification of Documents Extracted from Images with Optical Character Recognition Methods
• Can you tell? SSNet -- a Sagittal Stratum-inspired Neural Network Framework for Sentiment Analysis
• Look Ahead Text Understanding and LLM Stitching
• TechTexC: Classification of Technical Texts using Convolution and Bidirectional Long Short Term Memory Network
• Does a Hybrid Neural Network based Feature Selection Model Improve Text Classification?
• Transformer-F: A Transformer network with effective methods for learning universal sentence representation
• Self-supervised Regularization for Text Classification
• Towards a Data-Driven Requirements Engineering Approach: Automatic Analysis of User Reviews
• An Empirical Study of Topic Transition in Dialogue
• Discriminatory Expressions to Produce Interpretable Models in Short Documents
• Costs to Consider in Adopting NLP for Your Business
• Graph Convolutional Network for Swahili News Classification