Combination of Domain Knowledge and Deep Learning for Sentiment Analysis of Short and Informal Messages on Social Media

Abstract

Sentiment analysis has been emerging recently as one of the major natural language processing (NLP) tasks in many applications. Especially, as social media channels (e.g. social networks or forums) have become significant sources for brands to observe user opinions about their products, this task is thus increasingly crucial. However, when applied with real data obtained from social media, we notice that there is a high volume of short and informal messages posted by users on those channels. This kind of data makes the existing works suffer from many difficulties to handle, especially ones using deep learning approaches. In this paper, we propose an approach to handle this problem. This work is extended from our previous work, in which we proposed to combine the typical deep learning technique of Convolutional Neural Networks with domain knowledge. The combination is used for acquiring additional training data augmentation and a more reasonable loss function. In this work, we further improve our architecture by various substantial enhancements, including negation-based data augmentation, transfer learning for word embeddings, the combination of word-level embeddings and character-level embeddings, and using multitask learning technique for attaching domain knowledge rules in the learning process. Those enhancements, specifically aiming to handle short and informal messages, help us to enjoy significant improvement in performance once experimenting on real datasets.

arXiv commentA Preprint of an article accepted for publication by Inderscience in IJCVR on September 2018

References in corpus (8)

• Efficient Estimation of Word Representations in Vector Space
• An overview of gradient descent optimization algorithms
• An Overview of Multi-Task Learning in Deep Neural Networks
• Thumbs up? Sentiment Classification using Machine Learning Techniques
• Thumbs Up or Thumbs Down? Semantic Orientation Applied to Unsupervised Classification of Reviews
• Ask Me Anything: Dynamic Memory Networks for Natural Language Processing
• Text Understanding from Scratch
• Combination of Domain Knowledge and Deep Learning for Sentiment Analysis