Explaining First Impressions: Modeling, Recognizing, and Explaining Apparent Personality from Videos

Abstract

Explainability and interpretability are two critical aspects of decision support systems. Within computer vision, they are critical in certain tasks related to human behavior analysis such as in health care applications. Despite their importance, it is only recently that researchers are starting to explore these aspects. This paper provides an introduction to explainability and interpretability in the context of computer vision with an emphasis on looking at people tasks. Specifically, we review and study those mechanisms in the context of first impressions analysis. To the best of our knowledge, this is the first effort in this direction. Additionally, we describe a challenge we organized on explainability in first impressions analysis from video. We analyze in detail the newly introduced data set, the evaluation protocol, and summarize the results of the challenge. Finally, derived from our study, we outline research opportunities that we foresee will be decisive in the near future for the development of the explainable computer vision field.

Preprint submitted to TAC

References in corpus (12)

• Adam: A Method for Stochastic Optimization
• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Understanding Neural Networks Through Deep Visualization
• Understanding Black-box Predictions via Influence Functions
• Visualizing Deep Convolutional Neural Networks Using Natural Pre-Images
• Visualizing Deep Neural Network Decisions: Prediction Difference Analysis
• Grad-CAM: Why did you say that?
• Aligning Books and Movies: Towards Story-like Visual Explanations by Watching Movies and Reading Books
• "What is Relevant in a Text Document?": An Interpretable Machine Learning Approach
• Investigating the influence of noise and distractors on the interpretation of neural networks
• Proceedings of the 2016 ICML Workshop on Human Interpretability in Machine Learning (WHI 2016)
• Proceedings of the 2018 ICML Workshop on Human Interpretability in Machine Learning (WHI 2018)

Cited by in corpus (8)

• Recent Trends in Deep Learning Based Personality Detection
• PersEmoN: A Deep Network for Joint Analysis of Apparent Personality, Emotion and Their Relationship
• First Impressions: A Survey on Vision-Based Apparent Personality Trait Analysis
• Multimodal Personality Recognition using Cross-Attention Transformer and Behaviour Encoding
• Inferred vs traditional personality assessment: are we predicting the same thing?
• Robust Regression via Deep Negative Correlation Learning
• Face to Purchase: Predicting Consumer Choices with Structured Facial and Behavioral Traits Embedding
• Background Hardly Matters: Understanding Personality Attribution in Deep Residual Networks