5 papers
AffectCodec: Emotion-Preserving Neural Speech Codec for Expressive Speech Modeling
Jiacheng Shi, Hongfei Du, Xinyuan Song +3
Neural speech codecs provide discrete representations for speech language models, but emotional cues are often degraded during quantization. Existing codecs mainly optimize acousti…
Enhancing Conversational TTS with Cascaded Prompting and ICL-Based Online Reinforcement Learning
Zhicheng Ouyang, Seong-Gyun Leem, Bach Viet Do +4
Conversational AI has made significant progress, yet generating expressive and controllable text-to-speech (TTS) remains challenging. Specifically, controlling fine-grained voice s…
Aligning Paralinguistic Understanding and Generation in Speech LLMs via Multi-Task Reinforcement Learning
Jingxiang Chen, Minseok Kim, Seong-Gyun Leem +13
Speech large language models (LLMs) observe paralinguistic cues such as prosody, emotion, and non-verbal sounds--crucial for intent understanding. However, leveraging these cues fa…
Describe Where You Are: Improving Noise-Robustness for Speech Emotion Recognition with Text Description of the Environment
Seong-Gyun Leem, Daniel Fulford, Jukka-Pekka Onnela +2
Speech emotion recognition (SER) systems often struggle in real-world environments, where ambient noise severely degrades their performance. This paper explores a novel approach th…
The MSP-Podcast Corpus
Carlos Busso, Reza Lotfian, Kusha Sridhar +9
The availability of large, high-quality emotional speech databases is essential for advancing speech emotion recognition (SER) in real-world scenarios. However, many existing datab…