Video-based Social Interaction Behavior Analysis with the Simulated Interaction Task for Children (Kids-SIT)

Abstract

Accurately quantifying children's social interaction behavior is part of understanding their cognitive and emotional development, as well as mental health conditions. Kids-SIT is a web-based tool designed to computationally analyse children's behaviors by engaging them in a standardized video conversation while their responses are video recorded. In a pre-registered study with 21 healthy children and 12 children diagnosed with social anxiety disorder (SAD), aged 9-14 years, we assess its potential as an accessible paradigm for automated analysis of children's social interaction behavior. We evaluate whether the Kids-SIT can elicit naturalistic interaction patterns in healthy children, and how well automatic feature extraction methods can detect these patterns. We analyse children's subjective impressions, verbal responses, and non-verbal behaviors. Non-verbal behaviors were manually annotated and, independently, automatically extracted using state-of-the-art methods. In an exploratory analysis, we further assess whether automatically extracted features can distinguish between children with and without SAD. Verbal responses and post-hoc impressions indicate that the Kids-SIT elicits natural social interaction behavior. Non-verbal behavior aligned with this pattern: children looked at their interaction partner most of the time, particularly while listening rather than speaking. Smiling and gazing toward the partner occurred more frequently during the person-directed liked and disliked parts of the conversation than during the picture-description phase. These patterns were captured by both annotations and computational methods. Automatically extracted features enabled above-chance differentiation between children with and without SAD. Our results underscore the potential of the Kids-SIT for analysing children's social interaction behavior, with applicability extending to clinical contexts.