A Convolutional Neural Network for Modelling Sentences

Abstract

The ability to accurately represent sentences is central to language understanding. We describe a convolutional architecture dubbed the Dynamic Convolutional Neural Network (DCNN) that we adopt for the semantic modelling of sentences. The network uses Dynamic k-Max Pooling, a global pooling operation over linear sequences. The network handles input sentences of varying length and induces a feature graph over the sentence that is capable of explicitly capturing short and long-range relations. The network does not rely on a parse tree and is easily applicable to any language. We test the DCNN in four experiments: small scale binary and multi-class sentiment prediction, six-way question classification and Twitter sentiment prediction by distant supervision. The network achieves excellent performance in the first three tasks and a greater than 25% error reduction in the last task with respect to the strongest baseline.

References in corpus (5)

• Improving neural networks by preventing co-adaptation of feature detectors
• Learning to Map Sentences to Logical Form: Structured Classification with Probabilistic Categorial Grammars
• Recurrent Convolutional Neural Networks for Discourse Compositionality
• Mathematical Foundations for a Compositional Distributional Model of Meaning
• Category-Theoretic Quantitative Compositional Distributional Models of Natural Language Semantics

Cited by in corpus (159)

• An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling
• Deep Learning Enabled Semantic Communication Systems
• Convolutional Neural Networks for Sentence Classification
• Recurrent Neural Network for Text Classification with Multi-Task Learning
• Convolutional Networks on Graphs for Learning Molecular Fingerprints
• Comparative Study of CNN and RNN for Natural Language Processing
• A Deep Relevance Matching Model for Ad-hoc Retrieval
• Deep Learning for Answer Sentence Selection
• Distilling Task-Specific Knowledge from BERT into Simple Neural Networks
• A Deeper Look into Sarcastic Tweets Using Deep Convolutional Neural Networks
• A Deep Reinforcement Learning Chatbot
• Representation learning for very short texts using weighted word embedding aggregation
• Long Short-Term Memory-Networks for Machine Reading
• DeepSphere: Efficient spherical Convolutional Neural Network with HEALPix sampling for cosmological applications
• Text Matching as Image Recognition
• ConvBERT: Improving BERT with Span-based Dynamic Convolution
• Adversarial Feature Matching for Text Generation
• Ultradense Word Embeddings by Orthogonal Transformation
• Grammar Variational Autoencoder
• Recent Advances in Deep Learning: An Overview
• Evaluating Two-Stream CNN for Video Classification
• A Survey on Text Classification: From Shallow to Deep Learning
• Linguistically Regularized LSTMs for Sentiment Classification
• PipeDream: Fast and Efficient Pipeline Parallel DNN Training
• Adversarial Multi-task Learning for Text Classification
• From Captions to Visual Concepts and Back
• Aspect Level Sentiment Classification with Deep Memory Network
• Combining Neural Networks and Log-linear Models to Improve Relation Extraction
• Stochastic Language Generation in Dialogue using Recurrent Neural Networks with Convolutional Sentence Reranking
• Applying Deep Learning to Answer Selection: A Study and An Open Task
• Dependency-based Convolutional Neural Networks for Sentence Embedding
• Combating Fake News: A Survey on Identification and Mitigation Techniques
• Do Convolutional Networks need to be Deep for Text Classification ?
• Applications of Online Deep Learning for Crisis Response Using Social Media Information
• Aspect Based Sentiment Analysis with Gated Convolutional Networks
• Deconvolutional Paragraph Representation Learning
• Text Classification using Capsules
• Discriminative Neural Sentence Modeling by Tree-Based Convolution
• Discriminative Topic Mining via Category-Name Guided Text Embedding
• Universal, Unsupervised (Rule-Based), Uncovered Sentiment Analysis
• Detection of Bars in Galaxies using a Deep Convolutional Neural Network
• Recursive Neural Conditional Random Fields for Aspect-based Sentiment Analysis
• ABCNN: Attention-Based Convolutional Neural Network for Modeling Sentence Pairs
• Learning from Real Users: Rating Dialogue Success with Neural Networks for Reinforcement Learning in Spoken Dialogue Systems
• Learning text representation using recurrent convolutional neural network with highway layers
• Natural Language Inference by Tree-Based Convolution and Heuristic Matching
• Multimodal Representation for Neural Code Search
• Learning to Compose Words into Sentences with Reinforcement Learning
• Topic-based Evaluation for Conversational Bots
• EmoNets: Multimodal deep learning approaches for emotion recognition in video
• Rapid Classification of Crisis-Related Data on Social Networks using Convolutional Neural Networks
• Sentence-State LSTM for Text Representation
• PatchNet: Hierarchical Deep Learning-Based Stable Patch Identification for the Linux Kernel
• Reinforcement learning for optimal error correction of toric codes
• Deep Multi-Task Learning with Shared Memory
• Sales Forecast in E-commerce using Convolutional Neural Network
• A Fast Unified Model for Parsing and Sentence Understanding
• Grouped Convolutional Neural Networks for Multivariate Time Series
• Description Based Text Classification with Reinforcement Learning
• Topic2Vec: Learning Distributed Representations of Topics
• Compositional Distributional Semantics with Long Short Term Memory
• Distance Metric Learning for Aspect Phrase Grouping
• Deriving Neural Architectures from Sequence and Graph Kernels
• Towards Empathetic Human-Robot Interactions
• Combining Convolution and Recursive Neural Networks for Sentiment Analysis
• AC-BLSTM: Asymmetric Convolutional Bidirectional LSTM Networks for Text Classification
• 1D-Convolutional Capsule Network for Hyperspectral Image Classification
• Multichannel Variable-Size Convolution for Sentence Classification
• Toward Mention Detection Robustness with Recurrent Neural Networks
• BreakingNews: Article Annotation by Image and Text Processing
• Neural Discourse Relation Recognition with Semantic Memory
• Compositionality decomposed: how do neural networks generalise?
• Knowledge Graph Representation with Jointly Structural and Textual Encoding
• Learning to Represent Words in Context with Multilingual Supervision
• Learning Generic Sentence Representations Using Convolutional Neural Networks
• Overcoming Language Variation in Sentiment Analysis with Social Attention
• Enhancing Rumor Detection in Social Media Using Dynamic Propagation Structures
• Pre-Trained Models: Past, Present and Future
• Distilling Word Embeddings: An Encoding Approach
• FlipDial: A Generative Model for Two-Way Visual Dialogue
• Modelling customer online behaviours with neural networks: applications to conversion prediction and advertising retargeting
• Deconvolution-Based Global Decoding for Neural Machine Translation
• Neural Discourse Structure for Text Categorization
• Modelling Interaction of Sentence Pair with coupled-LSTMs
• Learning Document Embeddings by Predicting N-grams for Sentiment Classification of Long Movie Reviews
• Learning to Remember Translation History with a Continuous Cache
• $k$-Nearest Neighbor Augmented Neural Networks for Text Classification
• Deep Learning Based Cryptographic Primitive Classification
• UTCNN: a Deep Learning Model of Stance Classificationon on Social Media Text
• Deep Style Match for Complementary Recommendation
• Brain Intelligence: Go Beyond Artificial Intelligence
• Word Embeddings for Sentiment Analysis: A Comprehensive Empirical Survey
• Distributed Representations for Compositional Semantics
• Dynamic Compositional Neural Networks over Tree Structure
• Context-Dependent Translation Selection Using Convolutional Neural Network
• DeepSQLi: Deep Semantic Learning for Testing SQL Injection
• Hierarchical Question Answering for Long Documents
• Automatic Catchphrase Extraction from Legal Case Documents via Scoring using Deep Neural Networks
• A Context-Aware User-Item Representation Learning for Item Recommendation
• iCapsNets: Towards Interpretable Capsule Networks for Text Classification
• Same Representation, Different Attentions: Shareable Sentence Representation Learning from Multiple Tasks
• Integrating Physiological Time Series and Clinical Notes with Deep Learning for Improved ICU Mortality Prediction
• An Attention-Gated Convolutional Neural Network for Sentence Classification
• Multi-task Learning over Graph Structures
• Estimation of Inter-Sentiment Correlations Employing Deep Neural Network Models
• A Regressive Convolution Neural network and Support Vector Regression Model for Electricity Consumption Forecasting
• Improving Medical Short Text Classification with Semantic Expansion Using Word-Cluster Embedding
• Targeted Advertising Based on Browsing History
• Article citation study: Context enhanced citation sentiment detection
• Exploring Different Dimensions of Attention for Uncertainty Detection
• Towards More Fine-grained and Reliable NLP Performance Prediction
• CIS at TAC Cold Start 2015: Neural Networks and Coreference Resolution for Slot Filling
• Product-Aware Answer Generation in E-Commerce Question-Answering
• Japanese Sentiment Classification using a Tree-Structured Long Short-Term Memory with Attention
• Sentiment Classification with Word Attention based on Weakly Supervised Learning with a Convolutional Neural Network
• Comparing Convolutional Neural Networks to Traditional Models for Slot Filling
• Semantic-Unit-Based Dilated Convolution for Multi-Label Text Classification
• Attentive Convolution: Equipping CNNs with RNN-style Attention Mechanisms
• Machine Learning Meets Natural Language Processing -- The story so far
• A Language-independent and Compositional Model for Personality Trait Recognition from Short Texts
• Transfer Learning for Multi-lingual Tasks -- a Survey
• Sentence Encoding with Tree-constrained Relation Networks
• Surprising properties of dropout in deep networks
• DialogAct2Vec: Towards End-to-End Dialogue Agent by Multi-Task Representation Learning
• Mapping distributional to model-theoretic semantic spaces: a baseline
• Isometric Transformation Invariant Graph-based Deep Neural Network
• Siamese convolutional networks based on phonetic features for cognate identification
• Neural Networks for Joint Sentence Classification in Medical Paper Abstracts
• Parameterized Neural Network Language Models for Information Retrieval
• Analysing the potential of seq-to-seq models for incremental interpretation in task-oriented dialogue
• Image-based Natural Language Understanding Using 2D Convolutional Neural Networks
• Learning to Embed Sentences Using Attentive Recursive Trees
• Mixture of Expert/Imitator Networks: Scalable Semi-supervised Learning Framework
• Deep learning bank distress from news and numerical financial data
• A Weakly Supervised Approach to Train Temporal Relation Classifiers and Acquire Regular Event Pairs Simultaneously
• Group Sparse CNNs for Question Classification with Answer Sets
• Bringing Semantic Structures to User Intent Detection in Online Medical Queries
• Paraphrase-Supervised Models of Compositionality
• geoGAT: Graph Model Based on Attention Mechanism for Geographic Text Classification
• Mimicking Human Process: Text Representation via Latent Semantic Clustering for Classification
• Task-specific Word Identification from Short Texts Using a Convolutional Neural Network
• Fast and Accurate Recognition of Chinese Clinical Named Entities with Residual Dilated Convolutions
• Automatic Question-Answering Using A Deep Similarity Neural Network
• An Event Correlation Filtering Method for Fake News Detection
• Pose-based Body Language Recognition for Emotion and Psychiatric Symptom Interpretation
• Learning to Represent Bilingual Dictionaries
• Job Detection in Twitter
• Maps Search Misspelling Detection Leveraging Domain-Augmented Contextual Representations
• Constraint-free Natural Image Reconstruction from fMRI Signals Based on Convolutional Neural Network
• Joint Learning of Distributed Representations for Images and Texts
• Deep Domain Adaptation under Deep Label Scarcity
• Hierarchical Neural Network for Extracting Knowledgeable Snippets and Documents
• Text Classification based on Multiple Block Convolutional Highways
• Learning Robust Representations of Text
• Leveraging Medical Sentiment to Understand Patients Health on Social Media
• A Deep Learning-Based Approach for Measuring the Domain Similarity of Persian Texts
• Emotion Correlation Mining Through Deep Learning Models on Natural Language Text
• Text Classification with Lexicon from PreAttention Mechanism
• Drug cell line interaction prediction