5 papers
SemConFlow: Semantic Grounding of Holistic Co-Speech Gesture Generation with Contrastive Flow-Matching
Lanmiao Liu, Esam Ghaleb, Aslı Ãzyürek +1
While the field of co-speech gesture generation has seen significant advances, producing holistic, semantically grounded gestures remains a challenge. Existing approaches rely on e…
DuoGesture: Neuro-Inspired and Biomechanically Informed Dual-Stream Co-Speech Gesture Generation
Ferdinand Paar, Lanmiao Liu, Aslı Ãzyürek +2
Co-speech gesture generation requires both semantic expressivity and biomechanically plausible rhythmic motion. Existing holistic gesture models mix lexically grounded semantic ges…
Spectrum Matching: a Unified Perspective for Superior Diffusability in Latent Diffusion
Mang Ning, Mingxiao Li, Le Zhang +4
In this paper, we study the diffusability (learnability) of variational autoencoders (VAE) in latent diffusion. First, we show that pixel-space diffusion trained with an MSE object…
SemGes: Semantics-aware Co-Speech Gesture Generation using Semantic Coherence and Relevance Learning
Lanmiao Liu, Esam Ghaleb, Aslı Ãzyürek +1
Creating a virtual avatar with semantically coherent gestures that are aligned with speech is a challenging task. Existing gesture generation research mainly focused on generating…
DCTdiff: Intriguing Properties of Image Generative Modeling in the DCT Space
Mang Ning, Mingxiao Li, Jianlin Su +6
This paper explores image modeling from the frequency space and introduces DCTdiff, an end-to-end diffusion generative paradigm that efficiently models images in the discrete cosin…