6 papers
OmniOPSD: Rationale-Privileged On-Policy Self-Distillation for Affective Computing
Zebang Cheng, Shuimu Chen, Boxue Yang +7
Reinforcement learning for multimodal large language models (MLLMs) is often hindered by severe reward sparsity in complex reasoning tasks. This challenge is particularly pronounce…
Emotion-LLaMAv2 and MMEVerse: A New Framework and Benchmark for Multimodal Emotion Understanding
Xiaojiang Peng, Jingyi Chen, Zebang Cheng +11
Understanding human emotions from multimodal signals poses a significant challenge in affective computing and human-robot interaction. While multimodal large language models (MLLMs…
Do Audio LLMs Really LISTEN, or Just Transcribe? Measuring Lexical vs. Acoustic Emotion Cues Reliance
Jingyi Chen, Zhimeng Guo, Jiyun Chun +3
Understanding emotion from speech requires sensitivity to both lexical and acoustic cues. However, it remains unclear whether large audio language models (LALMs) genuinely process…
Equinox: Holistic Fair Scheduling in Serving Large Language Models
Zhixiang Wei, James Yen, Jingyi Chen +11
We address the limitations of current LLM serving with a dual-counter framework separating user and operator perspectives. The User Fairness Counter measures quality of service via…
Fine-Tuning Text-to-Speech Diffusion Models Using Reinforcement Learning with Human Feedback
Jingyi Chen, Ju Seung Byun, Micha Elsner +2
Diffusion models produce high-fidelity speech but are inefficient for real-time use due to long denoising steps and challenges in modeling intonation and rhythm. To improve this, w…
DLPO: Diffusion Model Loss-Guided Reinforcement Learning for Fine-Tuning Text-to-Speech Diffusion Models
Jingyi Chen, Ju-Seung Byun, Micha Elsner +1
Recent advancements in generative models have sparked a significant interest within the machine learning community. Particularly, diffusion models have demonstrated remarkable capa…