10 papers
A Comprehensive Study of Content Representations for Speech Synthesis
Diego Torres, Axel Roebel, Nicolas Obin
Speech content representations are central to voice conversion, speech-to-speech translation, and multimodal language models, yet they are rarely compared under a common generative…
Understanding Hyperspherical Geometry of ECAPA-TDNN Embedding and Its Impact on Zero-Shot Voice Conversion
Mathilde Abrassart, Nicolas Obin, Axel Roebel
Angular-margin speaker encoders are widely used in voice conversion, yet the geometry of their classifier prototypes remains poorly understood. We analyze ECAPA-TDNN classifier pro…
PitchFlower: A flow-based neural audio codec with pitch controllability
Diego Torres, Axel Roebel, Nicolas Obin
We present PitchFlower, a flow-based neural audio codec with explicit pitch controllability. Our approach promotes pitch disentanglement through a simple perturbation: during train…
Gelina: Unified Speech and Gesture Synthesis via Interleaved Token Prediction
Téo Guichoux, Théodor Lemerle, Shivam Mehta +5
Human communication is multimodal, with speech and gestures tightly coupled, yet most computational methods for generating speech and gestures synthesize them sequentially, weakeni…
Learning Relationships Between Separate Audio Tracks for Creative Applications
Balthazar Bujard, Jérôme Nika, Fédéric Bevilacqua +1
This paper presents the first step in a research project situated within the field of musical agents. The objective is to achieve, through training, the tuning of the desired music…
Lina-Speech: Gated Linear Attention and Initial-State Tuning for Multi-Sample Prompting Text-To-Speech Synthesis
Théodor Lemerle, Téo Guichoux, Axel Roebel +1
Neural codec language models, built on transformer architecture, have revolutionized text-to-speech (TTS) synthesis, excelling in voice cloning by treating it as a prefix continuat…
2D or not 2D: How Does the Dimensionality of Gesture Representation Affect 3D Co-Speech Gesture Generation?
Téo Guichoux, Laure Soulier, Nicolas Obin +1
Co-speech gestures are fundamental for communication. The advent of recent deep learning techniques has facilitated the creation of lifelike, synchronous co-speech gestures for Emb…
Investigating the impact of 2D gesture representation on co-speech gesture generation
Teo Guichoux, Laure Soulier, Nicolas Obin +1
Co-speech gestures play a crucial role in the interactions between humans and embodied conversational agents (ECA). Recent deep learning methods enable the generation of realistic,…
BWSNet: Automatic Perceptual Assessment of Audio Signals
Clément Le Moine Veillon, Victor Rosi, Pablo Arias Sarah +2
This paper introduces BWSNet, a model that can be trained from raw human judgements obtained through a Best-Worst scaling (BWS) experiment. It maps sound samples into an embedded s…
TranSTYLer: Multimodal Behavioral Style Transfer for Facial and Body Gestures Generation
Mireille Fares, Catherine Pelachaud, Nicolas Obin
This paper addresses the challenge of transferring the behavior expressivity style of a virtual agent to another one while preserving behaviors shape as they carry communicative me…