Sentiment Analysis with Contextual Embeddings and Self-Attention

Abstract

In natural language the intended meaning of a word or phrase is often implicit and depends on the context. In this work, we propose a simple yet effective method for sentiment analysis using contextual embeddings and a self-attention mechanism. The experimental results for three languages, including morphologically rich Polish and German, show that our model is comparable to or even outperforms state-of-the-art models. In all cases the superiority of models leveraging contextual embeddings is demonstrated. Finally, this work is intended as a step towards introducing a universal, multilingual sentiment classifier.

arXiv commentAccepted at the 25th International Symposium on Methodologies for Intelligent Systems (ISMIS 2020)

References in corpus (8)

• Distributed Representations of Words and Phrases and their Compositionality
• Neural Machine Translation by Jointly Learning to Align and Translate
• From Frequency to Meaning: Vector Space Models of Semantics
• Thumbs up? Sentiment Classification using Machine Learning Techniques
• Thumbs Up or Thumbs Down? Semantic Orientation Applied to Unsupervised Classification of Reviews
• A Structured Self-attentive Sentence Embedding
• Ask Me Anything: Dynamic Memory Networks for Natural Language Processing
• Attentive Pooling Networks

Cited by in corpus (2)

• Domain Adaptation of Transformer-Based Models using Unlabeled Data for Relevance and Polarity Classification of German Customer Feedback
• Re-Evaluating GermEval17 Using German Pre-Trained Language Models