A Data-driven Neural Network Architecture for Sentiment Analysis

Abstract

The fabulous results of convolution neural networks in image-related tasks, attracted attention of text mining, sentiment analysis and other text analysis researchers. It is however difficult to find enough data for feeding such networks, optimize their parameters, and make the right design choices when constructing network architectures. In this paper we present the creation steps of two big datasets of song emotions. We also explore usage of convolution and max-pooling neural layers on song lyrics, product and movie review text datasets. Three variants of a simple and flexible neural network architecture are also compared. Our intention was to spot any important patterns that can serve as guidelines for parameter optimization of similar models. We also wanted to identify architecture design choices which lead to high performing sentiment analysis models. To this end, we conducted a series of experiments with neural architectures of various configurations. Our results indicate that parallel convolutions of filter lengths up to three are usually enough for capturing relevant text features. Also, max-pooling region size should be adapted to the length of text documents for producing the best feature maps. Top results we got are obtained with feature maps of lengths 6 to 18. An improvement on future neural network models for sentiment analysis, could be generating sentiment polarity prediction of documents using aggregation of predictions on smaller excerpt of the entire text.

18 pages, 4 tables, 7 figures

References in corpus (7)

• Natural Language Processing (almost) from Scratch
• Striving for Simplicity: The All Convolutional Net
• Convolutional Neural Networks for Sentence Classification
• Seeing stars: Exploiting class relationships for sentiment categorization with respect to rating scales
• Text Classification Improved by Integrating Bidirectional LSTM with Two-dimensional Max Pooling
• Effective Use of Word Order for Text Categorization with Convolutional Neural Networks
• Convolutional Neural Networks for Text Categorization: Shallow Word-level vs. Deep Character-level

Cited by in corpus (2)

• Two Huge Title and Keyword Generation Corpora of Research Articles
• Automating Text Naturalness Evaluation of NLG Systems