Building a Large Scale Dataset for Image Emotion Recognition: The Fine Print and The Benchmark

Abstract

Psychological research results have confirmed that people can have different emotional reactions to different visual stimuli. Several papers have been published on the problem of visual emotion analysis. In particular, attempts have been made to analyze and predict people's emotional reaction towards images. To this end, different kinds of hand-tuned features are proposed. The results reported on several carefully selected and labeled small image data sets have confirmed the promise of such features. While the recent successes of many computer vision related tasks are due to the adoption of Convolutional Neural Networks (CNNs), visual emotion analysis has not achieved the same level of success. This may be primarily due to the unavailability of confidently labeled and relatively large image data sets for visual emotion analysis. In this work, we introduce a new data set, which started from 3+ million weakly labeled images of different emotions and ended up 30 times as large as the current largest publicly available visual emotion data set. We hope that this data set encourages further research on visual emotion analysis. We also perform extensive benchmarking analyses on this large data set using the state of the art methods including CNNs.

arXiv comment7 pages, 7 figures, AAAI 2016

References in corpus (3)

• Caffe: Convolutional Architecture for Fast Feature Embedding
• Fully Convolutional Networks for Semantic Segmentation
• Robust Image Sentiment Analysis Using Progressively Trained and Domain Transferred Deep Networks

Cited by in corpus (10)

• Learning Multi-level Deep Representations for Image Emotion Classification
• Multimodal Classification for Analysing Social Media
• Multi-source Domain Adaptation for Visual Sentiment Classification
• Contemplating Visual Emotions: Understanding and Overcoming Dataset Bias
• Visual-Texual Emotion Analysis with Deep Coupled Video and Danmu Neural Networks
• Pose-based Body Language Recognition for Emotion and Psychiatric Symptom Interpretation
• When Saliency Meets Sentiment: Understanding How Image Content Invokes Emotion and Sentiment
• Spatio-Temporal Sentiment Hotspot Detection Using Geotagged Photos
• Sentiment Recognition in Egocentric Photostreams
• Dual Pattern Learning Networks by Empirical Dual Prediction Risk Minimization