F-formation Detection: Individuating Free-standing Conversational Groups in Images

Abstract

Detection of groups of interacting people is a very interesting and useful task in many modern technologies, with application fields spanning from video-surveillance to social robotics. In this paper we first furnish a rigorous definition of group considering the background of the social sciences: this allows us to specify many kinds of group, so far neglected in the Computer Vision literature. On top of this taxonomy, we present a detailed state of the art on the group detection algorithms. Then, as a main contribution, we present a brand new method for the automatic detection of groups in still images, which is based on a graph-cuts framework for clustering individuals; in particular we are able to codify in a computational sense the sociological definition of F-formation, that is very useful to encode a group having only proxemic information: position and orientation of people. We call the proposed method Graph-Cuts for F-formation (GCFF). We show how GCFF definitely outperforms all the state of the art methods in terms of different accuracy measures (some of them are brand new), demonstrating also a strong robustness to noise and versatility in recognizing groups of various cardinality.

32 pages, submitted to PLOS One

Cited by in corpus (7)

• Deep Learning for Video Classification and Captioning
• Towards Storytelling from Visual Lifelogging: An Overview
• Improving Social Awareness Through DANTE: A Deep Affinity Network for Clustering Conversational Interactants
• Detecting socially interacting groups using f-formation: A survey of taxonomy, methods, datasets, applications, challenges, and future research directions
• Conversational Group Detection With Deep Convolutional Networks
• Learning Social Cost Functions for Human-Aware Path Planning
• SALSA: A Novel Dataset for Multimodal Group Behavior Analysis