5 papers
DuplexAct: How Interactive Are Frontier Full-Duplex Spoken Language Models?
Jiaheng Dong, Shihong Tan, Gongping Huang +3
End-to-end (E2E) full-duplex spoken language models (FD-SLMs) enable simutanous listening and speaking continously, requiring models to take appropriate interaction actions as conv…
Listen Then Reason: Perception-Grounded Test-Time Reinforcement Learning for Large Audio-Language Models
Jiaheng Dong, Xiaofeng Yu, Jean Honorio +3
Large audio-language models (LALMs) are increasingly used for a broader range of audio reasoning tasks. These models typically incorporate audio representations into a large langua…
Intermediate Text Representation Guided Text-to-Image Generation for Enhancing One-and-Only Alignment
Soyoun Won, Aryan Yazdan Parast, Basim Azam +2
Text-to-image (T2I) diffusion models often fail to faithfully render explicit textual descriptions, instead defaulting to strongly learned visual priors due to a phenomenon referre…
Disentangling Reasoning in Large Audio-Language Models for Ambiguous Emotion Prediction
Xiaofeng Yu, Jiaheng Dong, Jean Honorio +3
Speech emotion recognition plays an important role in various applications. However, most existing approaches predict a single emotion label, oversimplifying the inherently ambiguo…
Partial Inference in Structured Prediction
Chuyang Ke, Jean Honorio
In this paper, we examine the problem of partial inference in the context of structured prediction. Using a generative model approach, we consider the task of maximizing a score fu…