MultiMediate'23: Engagement Estimation and Bodily Behaviour Recognition in Social Interactions

Abstract

Automatic analysis of human behaviour is a fundamental prerequisite for the creation of machines that can effectively interact with- and support humans in social interactions. In MultiMediate'23, we address two key human social behaviour analysis tasks for the first time in a controlled challenge: engagement estimation and bodily behaviour recognition in social interactions. This paper describes the MultiMediate'23 challenge and presents novel sets of annotations for both tasks. For engagement estimation we collected novel annotations on the NOvice eXpert Interaction (NOXI) database. For bodily behaviour recognition, we annotated test recordings of the MPIIGroupInteraction corpus with the BBSI annotation scheme. In addition, we present baseline results for both challenge tasks.

arXiv commentACM MultiMedia'23

References in corpus (5)

• Modeling Engagement in Long-Term, In-Home Socially Assistive Robot Interventions for Children with Autism Spectrum Disorders
• Detecting Low Rapport During Natural Interactions in Small Groups from Non-Verbal Behaviour
• Bodily Behaviors in Social Interaction: Novel Annotations and State-of-the-Art Evaluation
• MultiMediate '22: Backchannel Detection and Agreement Estimation in Group Interactions
• Developing an Effective and Automated Patient Engagement Estimator for Telehealth: A Machine Learning Approach

Cited by in corpus (4)

• Data Augmentation for Human Behavior Analysis in Multi-Person Conversations
• Multilingual Dyadic Interaction Corpus NoXi+J: Toward Understanding Asian-European Non-verbal Cultural Characteristics and their Influences on Engagement
• MultiMediate'24: Multi-Domain Engagement Estimation
• MAGIC-TBR: Multiview Attention Fusion for Transformer-based Bodily Behavior Recognition in Group Settings