4 papers
Enhancing Conversational TTS with Cascaded Prompting and ICL-Based Online Reinforcement Learning
Zhicheng Ouyang, Seong-Gyun Leem, Bach Viet Do +4
Conversational AI has made significant progress, yet generating expressive and controllable text-to-speech (TTS) remains challenging. Specifically, controlling fine-grained voice s…
Aligning Paralinguistic Understanding and Generation in Speech LLMs via Multi-Task Reinforcement Learning
Jingxiang Chen, Minseok Kim, Seong-Gyun Leem +13
Speech large language models (LLMs) observe paralinguistic cues such as prosody, emotion, and non-verbal sounds--crucial for intent understanding. However, leveraging these cues fa…
Universal Semantic Disentangled Privacy-preserving Speech Representation Learning
Biel Tura Vecino, Subhadeep Maji, Aravind Varier +11
The use of audio recordings of human speech to train LLMs poses privacy concerns due to these models' potential to generate outputs that closely resemble artifacts in the training…
CA-SSLR: Condition-Aware Self-Supervised Learning Representation for Generalized Speech Processing
Yen-Ju Lu, Jing Liu, Thomas Thebaud +4
We introduce Condition-Aware Self-Supervised Learning Representation (CA-SSLR), a generalist conditioning model broadly applicable to various speech-processing tasks. Compared to s…