189 citations · 1.4k across the 55 of their papers we have counts for
11 papers · 1 filter
Integrated Speech and Gesture Synthesis
Siyang Wang, Simon Alexanderson, Joakim Gustafson +3
Text-to-speech and co-speech gesture synthesis have until now been treated as separate areas by two different research communities, and applications merely stack the two technologi…
Neural HMMs are all you need (for high-quality attention-free TTS)
Shivam Mehta, Éva Székely, Jonas Beskow +1
Neural sequence-to-sequence TTS has achieved significantly better output quality than statistical speech synthesis using HMMs. However, neural TTS is generally not probabilistic an…
Multimodal analysis of the predictability of hand-gesture properties
Taras Kucherenko, Rajmund Nagy, Michael Neff +2
Embodied conversational agents benefit from being able to accompany their speech with gestures. Although many data-driven approaches to gesture generation have been proposed in rec…
Normalizing Flow based Hidden Markov Models for Classification of Speech Phones with Explainability
Anubhab Ghosh, Antoine Honoré, Dong Liu +2
In pursuit of explainability, we develop generative models for sequential data. The proposed models provide state-of-the-art classification results and robust performance for speec…
The Case for Translation-Invariant Self-Attention in Transformer-Based Language Models
Ulme Wennberg, Gustav Eje Henter
Mechanisms for encoding positional information are central for transformer-based language models. In this paper, we analyze the position embeddings of existing language models, fin…
Speech2Properties2Gestures: Gesture-Property Prediction as a Tool for Generating Representational Gestures from Speech
Taras Kucherenko, Rajmund Nagy, Patrik Jonell +3
We propose a new framework for gesture generation, aiming to allow data-driven approaches to produce more semantically rich gestures. Our approach first predicts whether to gesture…
Transflower: probabilistic autoregressive dance generation with multimodal attention
Guillermo Valle-Pérez, Gustav Eje Henter, Jonas Beskow +3
Dance requires skillful composition of complex movements that follow rhythmic, tonal and timbral features of music. Formally, generating dance conditioned on a piece of music can b…
A large, crowdsourced evaluation of gesture generation systems on common data: The GENEA Challenge 2020
Taras Kucherenko, Patrik Jonell, Youngwoo Yoon +2
Co-speech gestures, gestures that accompany speech, play an important role in human communication. Automatic co-speech gesture generation is thus a key enabling technology for embo…
Robust Classification using Hidden Markov Models and Mixtures of Normalizing Flows
Anubhab Ghosh, Antoine Honoré, Dong Liu +2
We test the robustness of a maximum-likelihood (ML) based classifier where sequential data as observation is corrupted by noise. The hypothesis is that a generative model, that com…
Generating coherent spontaneous speech and gesture from text
Simon Alexanderson, Éva Székely, Gustav Eje Henter +2
Embodied human communication encompasses both verbal (speech) and non-verbal information (e.g., gesture and head movements). Recent advances in machine learning have substantially…
HEMVIP: Human Evaluation of Multiple Videos in Parallel
Patrik Jonell, Youngwoo Yoon, Pieter Wolfert +2
In many research areas, for example motion and gesture generation, objective measures alone do not provide an accurate impression of key stimulus traits such as perceived quality o…