Short Text Classification via Knowledge powered Attention with Similarity Matrix based CNN

Abstract

Short text is becoming more and more popular on the web, such as Chat Message, SMS and Product Reviews. Accurately classifying short text is an important and challenging task. A number of studies have difficulties in addressing this problem because of the word ambiguity and data sparsity. To address this issue, we propose a knowledge powered attention with similarity matrix based convolutional neural network (KASM) model, which can compute comprehensive information by utilizing the knowledge and deep neural network. We use knowledge graph (KG) to enrich the semantic representation of short text, specially, the information of parent-entity is introduced in our model. Meanwhile, we consider the word interaction in the literal-level between short text and the representation of label, and utilize similarity matrix based convolutional neural network (CNN) to extract it. For the purpose of measuring the importance of knowledge, we introduce the attention mechanisms to choose the important information. Experimental results on five standard datasets show that our model significantly outperforms state-of-the-art methods.

arXiv commentthere is an error in this paper

References in corpus (10)

• Character-level Convolutional Networks for Text Classification
• A Structured Self-attentive Sentence Embedding
• Recurrent Neural Network for Text Classification with Multi-Task Learning
• Comparative Study of CNN and RNN for Natural Language Processing
• Seeing stars: Exploiting class relationships for sentiment categorization with respect to rating scales
• TransA: An Adaptive Approach for Knowledge Graph Embedding
• Multilingual Hierarchical Attention Networks for Document Classification
• Question Answering over Freebase via Attentive RNN with Similarity Matrix based CNN
• Deep Short Text Classification with Knowledge Powered Attention
• Differentiating Concepts and Instances for Knowledge Graph Embedding