An Attention-Gated Convolutional Neural Network for Sentence Classification
arXiv:1808.07325
Abstract
The classification of sentences is very challenging, since sentences contain the limited contextual information. In this paper, we proposed an Attention-Gated Convolutional Neural Network (AGCNN) for sentence classification, which generates attention weights from the feature's context windows of different sizes by using specialized convolution encoders. It makes full use of limited contextual information to extract and enhance the influence of important features in predicting the sentence's category. Experimental results demonstrated that our model can achieve up to 3.1% higher accuracy than standard CNN models, and gain competitive results over the baselines on four out of the six tasks. Besides, we designed an activation function, namely, Natural Logarithm rescaled Rectified Linear Unit (NLReLU). Experiments showed that NLReLU can outperform ReLU and is comparable to other well-known activation functions on AGCNN.
Accepted for publication in the Intelligent Data Analysis journal, 19 pages, 4 figures and 5 tables
References in corpus (13)
- Efficient Estimation of Word Representations in Vector Space
- Natural Language Processing (almost) from Scratch
- Character-level Convolutional Networks for Text Classification
- Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)
- Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification
- Recurrent Neural Network for Text Classification with Multi-Task Learning
- Recurrent Models of Visual Attention
- Seeing stars: Exploiting class relationships for sentiment categorization with respect to rating scales
- Self-Normalizing Neural Networks
- A Brief Survey of Text Mining: Classification, Clustering and Extraction Techniques
- A Convolutional Neural Network for Modelling Sentences
- Word Embedding Perturbation for Sentence Classification
- Dynamic Compositional Neural Networks over Tree Structure