A Benchmark Study on Sentiment Analysis for Software Engineering Research

Abstract

A recent research trend has emerged to identify developers' emotions, by applying sentiment analysis to the content of communication traces left in collaborative development environments. Trying to overcome the limitations posed by using off-the-shelf sentiment analysis tools, researchers recently started to develop their own tools for the software engineering domain. In this paper, we report a benchmark study to assess the performance and reliability of three sentiment analysis tools specifically customized for software engineering. Furthermore, we offer a reflection on the open challenges, as they emerge from a qualitative analysis of misclassified texts.

Proceedings of 15th International Conference on Mining Software Repositories (MSR 2018)

References in corpus (5)

• Machine Learning in Automated Text Categorization
• Sentiment Polarity Detection for Software Development
• Mining Valence, Arousal, and Dominance - Possibilities for Detecting Burnout and Productivity?
• Recommending Insightful Comments for Source Code using Crowdsourced Knowledge
• Bootstrapping a Lexicon for Emotional Arousal in Software Engineering

Cited by in corpus (15)

• Deep Learning-based Sentiment Classification: A Comparative Survey
• BERT-Based Sentiment Analysis: A Software Engineering Perspective
• Sentiment Analysis Tools in Software Engineering: A Systematic Mapping Study
• SEntiMoji: An Emoji-Powered Learning Approach for Sentiment Analysis in Software Engineering
• Revisiting Sentiment Analysis for Software Engineering in the Era of Large Language Models
• Development and Application of Sentiment Analysis Tools in Software Engineering: A Systematic Literature Review
• Using Experience Sampling to link Software Repositories with Emotions and Work Well-Being
• Assessment of Off-the-Shelf SE-specific Sentiment Analysis Tools: An Extended Replication Study
• TOUR: Dynamic Topic and Sentiment Analysis of User Reviews for Assisting App Release
• How angry are your customers? Sentiment analysis of support tickets that escalate
• Love, Joy, Anger, Sadness, Fear, and Surprise: SE Needs Special Kinds of AI: A Case Study on Text Mining and SE
• Are Prompts All You Need? Evaluating Prompt-Based Large Language Models (LLM)s for Software Requirements Classification
• On the Limitations of Combining Sentiment Analysis Tools in a Cross-Platform Setting
• Individual Differences Limit Predicting Well-being and Productivity Using Software Repositories: A Longitudinal Industrial Study
• Sentiment analysis for software engineering: How far can zero-shot learning (ZSL) go?