Cats and Captions vs. Creators and the Clock: Comparing Multimodal Content to Context in Predicting Relative Popularity

Abstract

The content of today's social media is becoming more and more rich, increasingly mixing text, images, videos, and audio. It is an intriguing research question to model the interplay between these different modes in attracting user attention and engagement. But in order to pursue this study of multimodal content, we must also account for context: timing effects, community preferences, and social factors (e.g., which authors are already popular) also affect the amount of feedback and reaction that social-media posts receive. In this work, we separate out the influence of these non-content factors in several ways. First, we focus on ranking pairs of submissions posted to the same community in quick succession, e.g., within 30 seconds, this framing encourages models to focus on time-agnostic and community-specific content features. Within that setting, we determine the relative performance of author vs. content features. We find that victory usually belongs to "cats and captions," as visual and textual features together tend to outperform identity-based features. Moreover, our experiments show that when considered in isolation, simple unigram text features and deep neural network visual features yield the highest accuracy individually, and that the combination of the two modalities generally leads to the best accuracies overall.

arXiv comment10 pages, data and models available at http://www.cs.cornell.edu/~jhessel/cats/cats.html, Proceedings of WWW 2017

References in corpus (11)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Evolution of Reddit: From the Front Page of the Internet to a Self-referential Community?
• You had me at hello: How phrasing affects memorability
• Understanding Image Virality
• On the Interplay between Social and Topical Structure
• All Who Wander: On the Prevalence and Characteristics of Multi-community Engagement
• How to Ask for a Favor: A Case Study on the Success of Altruistic Requests
• Science, AskScience, and BadScience: On the Coexistence of Highly Related Communities
• Time Matters: Multi-scale Temporalization of Social Media Popularity
• Images Don't Lie: Transferring Deep Visual Semantic Features to Large-Scale Multimodal Learning to Rank
• Item Popularity Prediction in E-commerce Using Image Quality Feature Vectors

Cited by in corpus (5)

• The anatomy of Reddit: An overview of academic research
• Caveat Emptor, Computational Social Science: Large-Scale Missing Data in a Widely-Published Reddit Corpus
• Intrinsic Image Popularity Assessment
• Salienteye: Maximizing Engagement While Maintaining Artistic Style on Instagram Using Deep Neural Networks
• How to Make Your Multi-Image Posts Popular? An Approach to Enhanced Grid for Nine Images on Social Media