Comparative Study of CNN and RNN for Natural Language Processing

Abstract

Deep neural networks (DNN) have revolutionized the field of natural language processing (NLP). Convolutional neural network (CNN) and recurrent neural network (RNN), the two main types of DNN architectures, are widely explored to handle various NLP tasks. CNN is supposed to be good at extracting position-invariant features and RNN at modeling units in sequence. The state of the art on many NLP tasks often switches due to the battle between CNNs and RNNs. This work is the first systematic comparison of CNN and RNN on a wide range of representative NLP tasks, aiming to give basic guidance for DNN selection.

7 pages, 11 figures

References in corpus (7)

• Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
• On the Properties of Neural Machine Translation: Encoder-Decoder Approaches
• Language Modeling with Gated Convolutional Networks
• A Convolutional Neural Network for Modelling Sentences
• A large annotated corpus for learning natural language inference
• Traversing Knowledge Graphs in Vector Space
• Learning text representation using recurrent convolutional neural network with highway layers

Cited by in corpus (85)

• An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling
• On Hyperparameter Optimization of Machine Learning Algorithms: Theory and Practice
• Recent Trends in Deep Learning Based Natural Language Processing
• A Gated Recurrent Unit Approach to Bitcoin Price Prediction
• Look Before You Leap: Detecting Phishing Web Pages by Exploiting Raw URL And HTML Characteristics
• Trends in deep learning for medical hyperspectral image analysis
• HTMLPhish: Enabling Phishing Web Page Detection by Applying Deep Learning Techniques on HTML Analysis
• Sarcasm Detection using Hybrid Neural Network
• A Survey of Deep Learning Methods for Relation Extraction
• Interactive Learning for Identifying Relevant Tweets to Support Real-time Situational Awareness
• Unsupervised Law Article Mining based on Deep Pre-Trained Language Representation Models with Application to the Italian Civil Code
• Decoding EEG Brain Activity for Multi-Modal Natural Language Processing
• Artificial Intelligence and Deep Learning Algorithms for Epigenetic Sequence Analysis: A Review for Epigeneticists and AI Experts
• Why Self-Attention? A Targeted Evaluation of Neural Machine Translation Architectures
• HeBERT & HebEMO: a Hebrew BERT Model and a Tool for Polarity Analysis and Emotion Recognition
• Legal Document Classification: An Application to Law Area Prediction of Petitions to Public Prosecution Service
• ClipGen: A Deep Generative Model for Clipart Vectorization and Synthesis
• Transformer Hawkes Process
• A deep learning pipeline for identification of motor units in musculoskeletal ultrasound
• LabelSens: Enabling Real-time Sensor Data Labelling at the point of Collection on Edge Computing
• Identifying tax evasion in Mexico with tools from network science and machine learning
• Quo Vadis: Hybrid Machine Learning Meta-Model based on Contextual and Behavioral Malware Representations
• Learning to Exploit Invariances in Clinical Time-Series Data using Sequence Transformer Networks
• Deep reinforcement learning for time series: playing idealized trading games
• Augmenting Neural Machine Translation with Knowledge Graphs
• Improving Natural Language Inference Using External Knowledge in the Science Questions Domain
• Detection of Illicit Drug Trafficking Events on Instagram: A Deep Multimodal Multilabel Learning Approach
• SCARL: Side-Channel Analysis with Reinforcement Learning on the Ascon Authenticated Cipher
• Seizure Prediction Using Bidirectional LSTM
• Toward a Period-Specific Optimized Neural Network for OCR Error Correction of Historical Hebrew Texts
• Learning Graph-Level Representations with Recurrent Neural Networks
• Literature review on vulnerability detection using NLP technology
• Deep Sequence Models for Text Classification Tasks
• ProtoryNet - Interpretable Text Classification Via Prototype Trajectories
• Classifying medical notes into standard disease codes using Machine Learning
• Unlocking the Power of Deep PICO Extraction: Step-wise Medical NER Identification
• Rethinking Recurrent Neural Networks and Other Improvements for Image Classification
• Short Text Classification via Knowledge powered Attention with Similarity Matrix based CNN
• Dialogue-Based Relation Extraction
• VEGA: Towards an End-to-End Configurable AutoML Pipeline
• Investigating the Effect of Segmentation Methods on Neural Model based Sentiment Analysis on Informal Short Texts in Turkish
• A Precisely Xtreme-Multi Channel Hybrid Approach For Roman Urdu Sentiment Analysis
• A Survey of Neural Networks and Formal Languages
• SeVeN: Augmenting Word Embeddings with Unsupervised Relation Vectors
• Assessing the Ability of Self-Attention Networks to Learn Word Order
• Universal Approximation of Input-Output Maps by Temporal Convolutional Nets
• Neural network approach to classifying alarming student responses to online assessment
• CNNs, LSTMs, and Attention Networks for Pathology Detection in Medical Data
• Machine Learning Meets Natural Language Processing -- The story so far
• Benchmark Performance of Machine And Deep Learning Based Methodologies for Urdu Text Document Classification
• Unified recurrent neural network for many feature types
• CRNNTL: convolutional recurrent neural network and transfer learning for QSAR modelling
• A Hierarchical Self-attentive Convolution Network for Review Modeling in Recommendation Systems
• Comparing Attention-based Convolutional and Recurrent Neural Networks: Success and Limitations in Machine Reading Comprehension
• On Modelling Label Uncertainty in Deep Neural Networks: Automatic Estimation of Intra-observer Variability in 2D Echocardiography Quality Assessment
• Comparison of Neural Network Architectures for Spectrum Sensing
• Predicting Heart Failure Readmission from Clinical Notes Using Deep Learning
• A Token-wise CNN-based Method for Sentence Compression
• A Markov Decision Process Approach to Active Meta Learning
• i6mA-CNN: a convolution based computational approach towards identification of DNA N6-methyladenine sites in rice genome
• Language Identification of Devanagari Poems
• Frequency Pooling: Shift-Equivalent and Anti-Aliasing Downsampling
• BERT Fine-Tuning for Sentiment Analysis on Indonesian Mobile Apps Reviews
• Deep Learning Paradigm with Transformed Monolingual Word Embeddings for Multilingual Sentiment Analysis
• Lightweight Transformer in Federated Setting for Human Activity Recognition
• Hybrid Tiled Convolutional Neural Networks for Text Sentiment Classification
• Patient ADE Risk Prediction through Hierarchical Time-Aware Neural Network Using Claim Codes
• Exploring Neural Net Augmentation to BERT for Question Answering on SQUAD 2.0
• Peak Alignment of Gas Chromatography-Mass Spectrometry Data with Deep Learning
• Term Definitions Help Hypernymy Detection
• CNNTOP: a CNN-based Trajectory Owner Prediction Method
• When deep learning models on GPU can be accelerated by taking advantage of unstructured sparsity
• Constructing a Highlight Classifier with an Attention Based LSTM Neural Network
• How May I Help You? Using Neural Text Simplification to Improve Downstream NLP Tasks
• Privacy-Preserving and Efficient Data Collection Scheme for AMI Networks Using Deep Learning
• PredCoin: Defense against Query-based Hard-label Attack
• Computationally Efficient Wasserstein Loss for Structured Labels
• dictNN: A Dictionary-Enhanced CNN Approach for Classifying Hate Speech on Twitter
• A Deep Learning-Based Approach for Measuring the Domain Similarity of Persian Texts
• CrowdTSC: Crowd-based Neural Networks for Text Sentiment Classification
• DrugDBEmbed : Semantic Queries on Relational Database using Supervised Column Encodings
• Cost-Quality Adaptive Active Learning for Chinese Clinical Named Entity Recognition
• REflex: Flexible Framework for Relation Extraction in Multiple Domains
• Towards Controlled Transformation of Sentiment in Sentences
• The emergent algebraic structure of RNNs and embeddings in NLP