Speech2Properties2Gestures: Gesture-Property Prediction as a Tool for Generating Representational Gestures from Speech

Abstract

We propose a new framework for gesture generation, aiming to allow data-driven approaches to produce more semantically rich gestures. Our approach first predicts whether to gesture, followed by a prediction of the gesture properties. Those properties are then used as conditioning for a modern probabilistic gesture-generation model capable of high-quality output. This empowers the approach to generate gestures that are both diverse and representational. Follow-ups and more information can be found on the project page: https://svito-zar.github.io/speech2properties2gestures/ .

Accepted for publication at the ACM International Conference on Intelligent Virtual Agents (IVA 2021)

References in corpus (3)

• Speech Gesture Generation from the Trimodal Context of Text, Audio, and Speaker Identity
• A large, crowdsourced evaluation of gesture generation systems on common data: The GENEA Challenge 2020
• Understanding the Predictability of Gesture Parameters from Speech and their Perceptual Importance