Comparative Study of CNN and RNN for Natural Language Processing

Abstract

Deep neural networks (DNN) have revolutionized the field of natural language processing (NLP). Convolutional neural network (CNN) and recurrent neural network (RNN), the two main types of DNN architectures, are widely explored to handle various NLP tasks. CNN is supposed to be good at extracting position-invariant features and RNN at modeling units in sequence. The state of the art on many NLP tasks often switches due to the battle between CNNs and RNNs. This work is the first systematic comparison of CNN and RNN on a wide range of representative NLP tasks, aiming to give basic guidance for DNN selection.

7 pages, 11 figures

References in corpus (4)

• Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
• On the Properties of Neural Machine Translation: Encoder-Decoder Approaches
• A Convolutional Neural Network for Modelling Sentences
• Learning text representation using recurrent convolutional neural network with highway layers

Cited by in corpus (42)

• A Gated Recurrent Unit Approach to Bitcoin Price Prediction
• A Survey of Deep Learning Methods for Relation Extraction
• ClipGen: A Deep Generative Model for Clipart Vectorization and Synthesis
• Augmenting Neural Machine Translation with Knowledge Graphs
• Detection of Illicit Drug Trafficking Events on Instagram: A Deep Multimodal Multilabel Learning Approach
• SCARL: Side-Channel Analysis with Reinforcement Learning on the Ascon Authenticated Cipher
• Seizure Prediction Using Bidirectional LSTM
• Literature review on vulnerability detection using NLP technology
• Unlocking the Power of Deep PICO Extraction: Step-wise Medical NER Identification
• Dialogue-Based Relation Extraction
• VEGA: Towards an End-to-End Configurable AutoML Pipeline
• Assessing the Ability of Self-Attention Networks to Learn Word Order
• A Survey of Neural Networks and Formal Languages
• Investigating the Effect of Segmentation Methods on Neural Model based Sentiment Analysis on Informal Short Texts in Turkish
• Benchmark Performance of Machine And Deep Learning Based Methodologies for Urdu Text Document Classification
• Machine Learning Meets Natural Language Processing -- The story so far
• CNNs, LSTMs, and Attention Networks for Pathology Detection in Medical Data
• CRNNTL: convolutional recurrent neural network and transfer learning for QSAR modelling
• On Modelling Label Uncertainty in Deep Neural Networks: Automatic Estimation of Intra-observer Variability in 2D Echocardiography Quality Assessment
• A Hierarchical Self-attentive Convolution Network for Review Modeling in Recommendation Systems
• CNNTOP: a CNN-based Trajectory Owner Prediction Method
• Comparison of Neural Network Architectures for Spectrum Sensing
• Hybrid Tiled Convolutional Neural Networks for Text Sentiment Classification
• Deep Learning Paradigm with Transformed Monolingual Word Embeddings for Multilingual Sentiment Analysis
• Predicting Heart Failure Readmission from Clinical Notes Using Deep Learning
• A Token-wise CNN-based Method for Sentence Compression
• A Markov Decision Process Approach to Active Meta Learning
• i6mA-CNN: a convolution based computational approach towards identification of DNA N6-methyladenine sites in rice genome
• Language Identification of Devanagari Poems
• Frequency Pooling: Shift-Equivalent and Anti-Aliasing Downsampling
• BERT Fine-Tuning for Sentiment Analysis on Indonesian Mobile Apps Reviews
• PredCoin: Defense against Query-based Hard-label Attack
• Constructing a Highlight Classifier with an Attention Based LSTM Neural Network
• DrugDBEmbed : Semantic Queries on Relational Database using Supervised Column Encodings
• Cost-Quality Adaptive Active Learning for Chinese Clinical Named Entity Recognition
• Privacy-Preserving and Efficient Data Collection Scheme for AMI Networks Using Deep Learning
• Towards Controlled Transformation of Sentiment in Sentences
• How May I Help You? Using Neural Text Simplification to Improve Downstream NLP Tasks
• A Deep Learning-Based Approach for Measuring the Domain Similarity of Persian Texts
• CrowdTSC: Crowd-based Neural Networks for Text Sentiment Classification
• dictNN: A Dictionary-Enhanced CNN Approach for Classifying Hate Speech on Twitter
• Computationally Efficient Wasserstein Loss for Structured Labels