Gesture Evaluation in Virtual Reality

Abstract

Gestures are central to human communication, enriching interactions through non-verbal expression. Virtual avatars increasingly use AI-generated gestures to enhance life-likeness, yet evaluations have largely been confined to 2D. Virtual Reality (VR) provides an immersive alternative that may affect how gestures are perceived. This paper presents a comparative evaluation of computer-generated gestures in VR and 2D, examining three models from the 2023 GENEA Challenge. Results show that gestures viewed in VR were rated slightly higher on average, with the strongest effect observed for motion-capture "true movement." While model rankings remained consistent across settings, VR influenced participants' overall perception and offered unique benefits over traditional 2D evaluation.

Published in Proceedings of the 26th International Conference on Multimodal Interaction (ICMI '24), ACM. Copyright 2024 ACM. Licensed under CC BY

References in corpus (5)

• A Review of Evaluation Practices of Gesture Generation in Embodied Conversational Agents
• The DiffuseStyleGesture+ entry to the GENEA Challenge 2023
• Diffusion-Based Co-Speech Gesture Generation Using Joint Text and Audio Representation
• HEMVIP: Human Evaluation of Multiple Videos in Parallel
• Learning to Generate Pointing Gestures in Situated Embodied Conversational Agents

Cited by in corpus (1)

• Evaluation of Generative Models for Emotional 3D Animation Generation in VR