5 citations · 5 across the 5 of their papers we have counts for
9 papers
Do Motion Tokenizers for Co-Speech Gesture Generation Encode Gesture Semantics?
Varsha Suresh, Divij Jain, Jia Liu +2
Discrete motion tokenizers encode motion as atomic units and are widely used for co-speech gesture generation. It remains unclear which motion properties, especially those relevant…
ReFree: Towards Realistic Co-Speech Video Generation via Reward-Free RL and Multilevel Speech Guidance
Salaheldin Mohamed, M. Hamza Mughal, Rishabh Dabral +1
Speech-driven talking character animation seeks to generate life-like portrait videos that convey natural conversation behavior, aligning facial motion with spoken audio. Although…
Semantic Motion Anchors: Bridging Motion and Meaning in Co-Speech Gestures
Varsha Suresh, Mohammad Mahdi Abootorabi, Mohamed Salman +5
Learning a shared representation between spoken text and gesture is central to co-speech gesture retrieval, synthesis, and understanding, but remains challenging for semantically m…
Towards Reliable Human Evaluations in Gesture Generation: Insights from a Community-Driven State-of-the-Art Benchmark
Rajmund Nagy, Hendric Voss, Thanh Hoang-Minh +18
We review human evaluation practices in automatic, speech-driven 3D gesture generation and find a lack of standardisation and frequent use of flawed experimental setups. This leads…
Modeling Turn-Taking with Semantically Informed Gestures
Varsha Suresh, M. Hamza Mughal, Christian Theobalt +1
In conversation, humans use multimodal cues, such as speech, gestures, and gaze, to manage turn-taking. While linguistic and acoustic features are informative, gestures provide com…
Enhancing Spoken Discourse Modeling in Language Models Using Gestural Cues
Varsha Suresh, M. Hamza Mughal, Christian Theobalt +1
Research in linguistics shows that non-verbal cues, such as gestures, play a crucial role in spoken discourse. For example, speakers perform hand gestures to indicate topic shifts,…
Retrieving Semantics from the Deep: an RAG Solution for Gesture Synthesis
M. Hamza Mughal, Rishabh Dabral, Merel C. J. Scholman +2
Non-verbal communication often comprises of semantically rich gestures that help convey the meaning of an utterance. Producing such semantic co-speech gestures has been a major cha…
ConvoFusion: Multi-Modal Conversational Diffusion for Co-Speech Gesture Synthesis
Muhammad Hamza Mughal, Rishabh Dabral, Ikhsanul Habibie +3
Gestures play a key role in human communication. Recent methods for co-speech gesture generation, while managing to generate beat-aligned motions, struggle generating gestures that…
MoFusion: A Framework for Denoising-Diffusion-based Motion Synthesis
Rishabh Dabral, Muhammad Hamza Mughal, Vladislav Golyanik +1
Conventional methods for human motion synthesis are either deterministic or struggle with the trade-off between motion diversity and motion quality. In response to these limitation…