189 citations · 1.4k across the 55 of their papers we have counts for
11 papers · 1 filter
Unified speech and gesture synthesis using flow matching
Shivam Mehta, Ruibo Tu, Simon Alexanderson +3
As text-to-speech technologies achieve remarkable naturalness in read-aloud tasks, there is growing interest in multimodal synthesis of verbal and non-verbal communicative behaviou…
Matcha-TTS: A fast TTS architecture with conditional flow matching
Shivam Mehta, Ruibo Tu, Jonas Beskow +2
We introduce Matcha-TTS, a new encoder-decoder architecture for speedy TTS acoustic modelling, trained using optimal-transport conditional flow matching (OT-CFM). This yields an OD…
The GENEA Challenge 2023: A large scale evaluation of gesture generation models in monadic and dyadic settings
Taras Kucherenko, Rajmund Nagy, Youngwoo Yoon +4
This paper reports on the GENEA Challenge 2023, in which participating teams built speech-driven gesture-generation systems using the same speech and motion dataset, followed by a…
On the Use of Self-Supervised Speech Representations in Spontaneous Speech Synthesis
Siyang Wang, Gustav Eje Henter, Joakim Gustafson +1
Self-supervised learning (SSL) speech representations learned from large amounts of diverse, mixed-quality speech data without transcriptions are gaining ground in many speech tech…
Speaker-independent neural formant synthesis
Pablo Pérez Zarazaga, Zofia Malisz, Gustav Eje Henter +1
We describe speaker-independent speech synthesis driven by a small set of phonetically meaningful speech parameters such as formant frequencies. The intention is to leverage deep-l…
Diff-TTSG: Denoising probabilistic integrated speech and gesture synthesis
Shivam Mehta, Siyang Wang, Simon Alexanderson +3
With read-aloud speech synthesis achieving high naturalness scores, there is a growing research interest in synthesising spontaneous speech. However, human spontaneous face-to-face…
A processing framework to access large quantities of whispered speech found in ASMR
Pablo Perez Zarazaga, Gustav Eje Henter, Zofia Malisz
Whispering is a ubiquitous mode of communication that humans use daily. Despite this, whispered speech has been poorly served by existing speech technology due to a shortage of res…
Evaluating gesture generation in a large-scale open challenge: The GENEA Challenge 2022
Taras Kucherenko, Pieter Wolfert, Youngwoo Yoon +4
This paper reports on the second GENEA Challenge to benchmark data-driven automatic co-speech gesture generation. Participating teams used the same speech and motion dataset to bui…
A Comparative Study of Self-Supervised Speech Representations in Read and Spontaneous TTS
Siyang Wang, Gustav Eje Henter, Joakim Gustafson +1
Recent work has explored using self-supervised learning (SSL) speech representations such as wav2vec2.0 as the representation medium in standard two-stage TTS, in place of conventi…
Context-specific kernel-based hidden Markov model for time series analysis
Carlos Puerto-Santana, Concha Bielza, Pedro Larrañaga +1
Traditional hidden Markov models have been a useful tool to understand and model stochastic dynamic data; in the case of non-Gaussian data, models such as mixture of Gaussian hidde…
A Comprehensive Review of Data-Driven Co-Speech Gesture Generation
Simbarashe Nyatsanga, Taras Kucherenko, Chaitanya Ahuja +2
Gestures that accompany speech are an essential part of natural and efficient embodied human communication. The automatic generation of such co-speech gestures is a long-standing p…