Effective Use of Word Order for Text Categorization with Convolutional Neural Networks

Abstract

Convolutional neural network (CNN) is a neural network that can make use of the internal structure of data such as the 2D structure of image data. This paper studies CNN on text categorization to exploit the 1D structure (namely, word order) of text data for accurate prediction. Instead of using low-dimensional word vectors as input as is often done, we directly apply CNN to high-dimensional text data, which leads to directly learning embedding of small text regions for use in classification. In addition to a straightforward adaptation of CNN from image to text, a simple but new variation which employs bag-of-word conversion in the convolution layer is proposed. An extension to combine multiple convolution layers is also explored for higher accuracy. The experiments demonstrate the effectiveness of our approach in comparison with state-of-the-art methods.

References in corpus (4)

• Improving neural networks by preventing co-adaptation of feature detectors
• Going Deeper with Convolutions
• Visualizing and Understanding Convolutional Networks
• Ensemble of Generative and Discriminative Techniques for Sentiment Analysis of Movie Reviews

Cited by in corpus (58)

• HDLTex: Hierarchical Deep Learning for Text Classification
• Text Understanding from Scratch
• Quasi-Recurrent Neural Networks
• Natural Language Processing Advancements By Deep Learning: A Survey
• SCALE-Sim: Systolic CNN Accelerator Simulator
• Weakly-Supervised Neural Text Classification
• Supervised and Semi-Supervised Text Categorization using LSTM for Region Embeddings
• Hierarchical Text Classification with Reinforced Label Assignment
• Multi-Label Text Classification using Attention-based Graph Neural Network
• Representation Learning for Natural Language Processing
• Revisiting LSTM Networks for Semi-Supervised Text Classification via Mixed Objective Function
• Analysis of Railway Accidents' Narratives Using Deep Learning
• Deep Learning Sentiment Analysis of Amazon.com Reviews and Ratings
• Multi-Label Classification of Patient Notes a Case Study on ICD Code Assignment
• An Improvement of Data Classification Using Random Multimodel Deep Learning (RMDL)
• Towards Scalable and Reliable Capsule Networks for Challenging NLP Applications
• Convolutional Neural Networks for Toxic Comment Classification
• SHAP values for Explaining CNN-based Text Classification Models
• Structural block driven - enhanced convolutional neural representation for relation extraction
• MGNC-CNN: A Simple Approach to Exploiting Multiple Word Embeddings for Sentence Classification
• Rationale-Augmented Convolutional Neural Networks for Text Classification
• Joint Text Embedding for Personalized Content-based Recommendation
• Fishing for Clickbaits in Social Images and Texts with Linguistically-Infused Neural Network Models
• Convolutional Neural Network: Text Classification Model for Open Domain Question Answering System
• SeMemNN: A Semantic Matrix-Based Memory Neural Network for Text Classification
• Learning Convolutional Text Representations for Visual Question Answering
• Review Helpfulness Assessment based on Convolutional Neural Network
• Disentangled Variational Auto-Encoder for Semi-supervised Learning
• A Data-driven Neural Network Architecture for Sentiment Analysis
• Syntactic Recurrent Neural Network for Authorship Attribution
• Active Discriminative Text Representation Learning
• GradiVeQ: Vector Quantization for Bandwidth-Efficient Gradient Aggregation in Distributed CNN Training
• Literature review on vulnerability detection using NLP technology
• Text as Environment: A Deep Reinforcement Learning Text Readability Assessment Model
• Convolutional Neural Networks for Sentiment Classification on Business Reviews
• An Empirical Study on Sentiment Classification of Chinese Review using Word Embedding
• Empirical Comparisons of CNN with Other Learning Algorithms for Text Classification in Legal Document Review
• Visual and Textual Sentiment Analysis Using Deep Fusion Convolutional Neural Networks
• Cross-lingual Distillation for Text Classification
• Light-Weighted CNN for Text Classification
• Learning Semantically Coherent and Reusable Kernels in Convolution Neural Nets for Sentence Classification
• Words are not Equal: Graded Weighting Model for building Composite Document Vectors
• Contextual and Position-Aware Factorization Machines for Sentiment Classification
• Article citation study: Context enhanced citation sentiment detection
• Adversarial Training Based Multi-Source Unsupervised Domain Adaptation for Sentiment Analysis
• Non-local Recurrent Neural Memory for Supervised Sequence Modeling
• FSMJ: Feature Selection with Maximum Jensen-Shannon Divergence for Text Categorization
• RISC-NN: Use RISC, NOT CISC as Neural Network Hardware Infrastructure
• CNN Application in Detection of Privileged Documents in Legal Document Review
• Hierarchical Text Classification Using Contrastive Learning Informed Path Guided Hierarchy
• Image-based Natural Language Understanding Using 2D Convolutional Neural Networks
• Investigating the Working of Text Classifiers
• Ranking-Based Autoencoder for Extreme Multi-label Classification
• Search Intelligence: Deep Learning For Dominant Category Prediction
• Text Classification with Lexicon from PreAttention Mechanism
• Strategies for Language Identification in Code-Mixed Low Resource Languages
• Twitter Sentiment on Affordable Care Act using Score Embedding
• Diagnosis and Analysis of Celiac Disease and Environmental Enteropathy on Biopsy Images using Deep Learning Approaches