8 papers
DuplexAct: How Interactive Are Frontier Full-Duplex Spoken Language Models?
Jiaheng Dong, Shihong Tan, Gongping Huang +3
End-to-end (E2E) full-duplex spoken language models (FD-SLMs) enable simutanous listening and speaking continously, requiring models to take appropriate interaction actions as conv…
Listen Then Reason: Perception-Grounded Test-Time Reinforcement Learning for Large Audio-Language Models
Jiaheng Dong, Xiaofeng Yu, Jean Honorio +3
Large audio-language models (LALMs) are increasingly used for a broader range of audio reasoning tasks. These models typically incorporate audio representations into a large langua…
RAIL: Rethinking Auditory Intelligence in Large Audio-Language Models with a CHC-Grounded Benchmark
Hongyu Jin, Siyi Wang, Yang Xiao +11
Humans process rich auditory environments through various cognitive capabilities such as audio perception, audio reasoning and memory. Despite recent progress in large audio-langua…
Disentangling Reasoning in Large Audio-Language Models for Ambiguous Emotion Prediction
Xiaofeng Yu, Jiaheng Dong, Jean Honorio +3
Speech emotion recognition plays an important role in various applications. However, most existing approaches predict a single emotion label, oversimplifying the inherently ambiguo…
Localizing and Editing Knowledge in Large Audio-Language Models
Sung Kyun Chung, Jiaheng Dong, Qiuchi Hu +3
Large Audio-Language Models (LALMs) have shown strong performance in speech understanding, making speech a natural interface for accessing factual information. Yet they are trained…
ImKWS: Test-Time Adaptation for Keyword Spotting with Class Imbalance
Hanyu Ding, Yang Xiao, Jiaheng Dong +1
Keyword spotting (KWS) identifies words for voice assistants, but environmental noise frequently reduces accuracy. Standard adaptation fixes this issue and strictly requires origin…
Test-Time Adaptation for Speech Emotion Recognition
Jiaheng Dong, Hong Jia, Ting Dang
The practical utility of Speech Emotion Recognition (SER) systems is undermined by their fragility to domain shifts, such as speaker variability, the distinction between acted and…
E-BATS: Efficient Backpropagation-Free Test-Time Adaptation for Speech Foundation Models
Jiaheng Dong, Hong Jia, Soumyajit Chatterjee +3
Speech Foundation Models encounter significant performance degradation when deployed in real-world scenarios involving acoustic domain shifts, such as background noise and speaker…