Crowdsourcing a Word-Emotion Association Lexicon

Abstract

Even though considerable attention has been given to the polarity of words (positive and negative) and the creation of large polarity lexicons, research in emotion analysis has had to rely on limited and small emotion lexicons. In this paper we show how the combined strength and wisdom of the crowds can be used to generate a large, high-quality, word-emotion and word-polarity association lexicon quickly and inexpensively. We enumerate the challenges in emotion annotation in a crowdsourcing scenario and propose solutions to address them. Most notably, in addition to questions about emotions associated with terms, we show how the inclusion of a word choice question can discourage malicious data entry, help identify instances where the annotator may not be familiar with the target term (allowing us to reject such annotations), and help obtain annotations at sense level (rather than at word level). We conducted experiments on how to formulate the emotion-annotation questions, and show that asking if a term is associated with an emotion leads to markedly higher inter-annotator agreement than that obtained by asking if a term evokes an emotion.

Cited by in corpus (13)

• Emotion Detection in Text: a Review
• Market Trend Prediction using Sentiment Analysis: Lessons Learned and Paths Forward
• A Multi-task Ensemble Framework for Emotion, Sentiment and Intensity Prediction
• Attention-based Modeling for Emotion Detection and Classification in Textual Conversations
• Folksonomication: Predicting Tags for Movies from Plot Synopses Using Emotion Flow Encoded Neural Network
• Studying Politeness across Cultures Using English Twitter and Mandarin Weibo
• Introducing Aspects of Creativity in Automatic Poetry Generation
• Even the Abstract have Colour: Consensus in Word-Colour Associations
• Predicting Opioid Relapse Using Social Media Data
• DENS: A Dataset for Multi-class Emotion Analysis
• Nearest neighbour approaches for Emotion Detection in Tweets
• PoKi: A Large Dataset of Poems by Children
• Room to Grow: Understanding Personal Characteristics Behind Self Improvement Using Social Media