23 citations · 72 across the 10 of their papers we have counts for
14 papers
Region-Aware Masking for Accent-Robust Cross-Lingual Text-to-Speech
Haoqi Li, Shivam Mehta, Ravi Teja Gadde +1
Accent leakage remains a critical challenge in cross-lingual zero-shot text-to-speech (TTS), where models inadvertently transfer both the speaker's timbre and source-language accen…
Align Then Reason: A Multimodal Lip-Sync Judge for Dubbing
Rui Liu, Bhavin Jawade, Haoqi Li +4
Dubbing quality control requires a reference-free judge that can determine whether a candidate text line matches a speaker's visible articulation in both content and timing, using…
Louder, Longer, Livelier: Acoustic Shortcuts and Underspecified Rationales in Speech LLM Judges
Mingyue Huo, Shivam Mehta, Bhavin Jawade +2
LLM-as-a-judge is widely used for evaluating text, but extending this paradigm to speech requires models to interpret acoustic as well as linguistic evidence. This introduces a mod…
SpeechCritic: Learning a Diagnostic Speech Judge from Limited Human Preferences
Mingyue Huo, Shivam Mehta, Bhavin Jawade +2
Human speech conveys rich perceptual information, such as emotion and speaker identity, yet most automatic speech quality judges reduce it to a single naturalness score. We study d…
Gelina: Unified Speech and Gesture Synthesis via Interleaved Token Prediction
Téo Guichoux, Théodor Lemerle, Shivam Mehta +5
Human communication is multimodal, with speech and gestures tightly coupled, yet most computational methods for generating speech and gestures synthesize them sequentially, weakeni…
Should you use a probabilistic duration model in TTS? Probably! Especially for spontaneous speech
Shivam Mehta, Harm Lameris, Rajiv Punmiya +3
Converting input symbols to output audio in TTS requires modelling the durations of speech sounds. Leading non-autoregressive (NAR) TTS models treat duration modelling as a regress…
Fake it to make it: Using synthetic data to remedy the data shortage in joint multimodal speech-and-gesture synthesis
Shivam Mehta, Anna Deichler, Jim O'Regan +4
Although humans engaged in face-to-face conversation simultaneously communicate both verbally and non-verbally, methods for joint and unified synthesis of speech audio and co-speec…
Unified speech and gesture synthesis using flow matching
Shivam Mehta, Ruibo Tu, Simon Alexanderson +3
As text-to-speech technologies achieve remarkable naturalness in read-aloud tasks, there is growing interest in multimodal synthesis of verbal and non-verbal communicative behaviou…
Diffusion-Based Co-Speech Gesture Generation Using Joint Text and Audio Representation
Anna Deichler, Shivam Mehta, Simon Alexanderson +1
This paper describes a system developed for the GENEA (Generation and Evaluation of Non-verbal Behaviour for Embodied Agents) Challenge 2023. Our solution builds on an existing dif…
Matcha-TTS: A fast TTS architecture with conditional flow matching
Shivam Mehta, Ruibo Tu, Jonas Beskow +2
We introduce Matcha-TTS, a new encoder-decoder architecture for speedy TTS acoustic modelling, trained using optimal-transport conditional flow matching (OT-CFM). This yields an OD…
Diff-TTSG: Denoising probabilistic integrated speech and gesture synthesis
Shivam Mehta, Siyang Wang, Simon Alexanderson +3
With read-aloud speech synthesis achieving high naturalness scores, there is a growing research interest in synthesising spontaneous speech. However, human spontaneous face-to-face…
Prosody-controllable spontaneous TTS with neural HMMs
Harm Lameris, Shivam Mehta, Gustav Eje Henter +2
Spontaneous speech has many affective and pragmatic functions that are interesting and challenging to model in TTS. However, the presence of reduced articulation, fillers, repetiti…
OverFlow: Putting flows on top of neural transducers for better TTS
Shivam Mehta, Ambika Kirkland, Harm Lameris +3
Neural HMMs are a type of neural transducer recently proposed for sequence-to-sequence modelling in text-to-speech. They combine the best features of classic statistical speech syn…
Neural HMMs are all you need (for high-quality attention-free TTS)
Shivam Mehta, Éva Székely, Jonas Beskow +1
Neural sequence-to-sequence TTS has achieved significantly better output quality than statistical speech synthesis using HMMs. However, neural TTS is generally not probabilistic an…