189 citations · 687 across the 14 of their papers we have counts for
14 papers
Towards Context-Aware Human-like Pointing Gestures with RL Motion Imitation
Anna Deichler, Siyang Wang, Simon Alexanderson +1
Pointing is a key mode of interaction with robots, yet most prior work has focused on recognition rather than generation. We present a motion capture dataset of human pointing gest…
Learning to Generate Pointing Gestures in Situated Embodied Conversational Agents
Anna Deichler, Siyang Wang, Simon Alexanderson +1
One of the main goals of robotics and intelligent agent research is to enable natural communication with humans in physically situated settings. While recent work has focused on ve…
Incorporating Spatial Awareness in Data-Driven Gesture Generation for Virtual Agents
Anna Deichler, Simon Alexanderson, Jonas Beskow
This paper focuses on enhancing human-agent communication by integrating spatial context into virtual agents' non-verbal behaviors, specifically gestures. Recent advances in co-spe…
Fake it to make it: Using synthetic data to remedy the data shortage in joint multimodal speech-and-gesture synthesis
Shivam Mehta, Anna Deichler, Jim O'Regan +4
Although humans engaged in face-to-face conversation simultaneously communicate both verbally and non-verbally, methods for joint and unified synthesis of speech audio and co-speec…
Unified speech and gesture synthesis using flow matching
Shivam Mehta, Ruibo Tu, Simon Alexanderson +3
As text-to-speech technologies achieve remarkable naturalness in read-aloud tasks, there is growing interest in multimodal synthesis of verbal and non-verbal communicative behaviou…
Diffusion-Based Co-Speech Gesture Generation Using Joint Text and Audio Representation
Anna Deichler, Shivam Mehta, Simon Alexanderson +1
This paper describes a system developed for the GENEA (Generation and Evaluation of Non-verbal Behaviour for Embodied Agents) Challenge 2023. Our solution builds on an existing dif…
Diff-TTSG: Denoising probabilistic integrated speech and gesture synthesis
Shivam Mehta, Siyang Wang, Simon Alexanderson +3
With read-aloud speech synthesis achieving high naturalness scores, there is a growing research interest in synthesising spontaneous speech. However, human spontaneous face-to-face…
Listen, Denoise, Action! Audio-Driven Motion Synthesis with Diffusion Models
Simon Alexanderson, Rajmund Nagy, Jonas Beskow +1
Diffusion models have experienced a surge of interest as highly expressive yet efficiently trainable probabilistic models. We show that these models are an excellent fit for synthe…
Integrated Speech and Gesture Synthesis
Siyang Wang, Simon Alexanderson, Joakim Gustafson +3
Text-to-speech and co-speech gesture synthesis have until now been treated as separate areas by two different research communities, and applications merely stack the two technologi…
Transflower: probabilistic autoregressive dance generation with multimodal attention
Guillermo Valle-Pérez, Gustav Eje Henter, Jonas Beskow +3
Dance requires skillful composition of complex movements that follow rhythmic, tonal and timbral features of music. Formally, generating dance conditioned on a piece of music can b…
Generating coherent spontaneous speech and gesture from text
Simon Alexanderson, Éva Székely, Gustav Eje Henter +2
Embodied human communication encompasses both verbal (speech) and non-verbal information (e.g., gesture and head movements). Recent advances in machine learning have substantially…
Robust model training and generalisation with Studentising flows
Simon Alexanderson, Gustav Eje Henter
Normalising flows are tractable probabilistic models that leverage the power of deep learning to describe a wide parametric family of distributions, all while remaining trainable u…
Gesticulator: A framework for semantically-aware speech-driven gesture generation
Taras Kucherenko, Patrik Jonell, Sanne van Waveren +4
During speech, people spontaneously gesticulate, which plays a key role in conveying information. Similarly, realistic co-speech gestures are crucial to enable natural and smooth i…
MoGlow: Probabilistic and controllable motion synthesis using normalising flows
Gustav Eje Henter, Simon Alexanderson, Jonas Beskow
Data-driven modelling and synthesis of motion is an active research area with applications that include animation, games, and social robotics. This paper introduces a new class of…