Contemplating Visual Emotions: Understanding and Overcoming Dataset Bias

Abstract

While machine learning approaches to visual emotion recognition offer great promise, current methods consider training and testing models on small scale datasets covering limited visual emotion concepts. Our analysis identifies an important but long overlooked issue of existing visual emotion benchmarks in the form of dataset biases. We design a series of tests to show and measure how such dataset biases obstruct learning a generalizable emotion recognition model. Based on our analysis, we propose a webly supervised approach by leveraging a large quantity of stock image data. Our approach uses a simple yet effective curriculum guided training strategy for learning discriminative emotion features. We discover that the models learned using our large scale stock image dataset exhibit significantly better generalization ability than the existing datasets without the manual collection of even a single label. Moreover, visual representation learned using our approach holds a lot of promise across a variety of tasks on different image and video datasets.

ECCV 2018

References in corpus (8)

• Training Convolutional Networks with Noisy Labels
• WebVision Database: Visual Learning and Understanding from Web Data
• DeepSentiBank: Visual Sentiment Concept Classification with Deep Convolutional Neural Networks
• Robust Image Sentiment Analysis Using Progressively Trained and Domain Transferred Deep Networks
• Building a Large Scale Dataset for Image Emotion Recognition: The Fine Print and The Benchmark
• Webly Supervised Learning of Convolutional Networks
• Building Emotional Machines: Recognizing Image Emotions through Deep Neural Networks
• Exploiting Multi-modal Curriculum in Noisy Web Data for Large-scale Concept Learning