5 papers
Entity Binding Failures in Speech LLM Reasoning: Diagnosis and Chain-of-Thought Intervention
Ming-Hao Hsu, Xiaohai Tian, Jun Zhang +1
Speech Large Language Models (SLLMs) underperform their text counterparts on complex reasoning. We reveal that this gap is not a uniform cognitive deficit. Evaluating two architect…
Geometric Alignment without Functional Equivalence: A Layer-wise Analysis of the Speech-Text Modality Gap
Ming-Hao Hsu, Xueyao Zhang, Xiaohai Tian +4
End-to-end speech large language models (LLMs) often answer the same question less accurately from speech than from text, even when the spoken content matches the text prompt. Exis…
Integrating Fine-Grained Audio-Visual Evidence for Robust Multimodal Emotion Reasoning
Zhixian Zhao, Wenjie Tian, Lei Xie
Multimodal emotion analysis is shifting from static classification to generative reasoning. Beyond simple label prediction, robust affective reasoning must synthesize fine-grained…
Selective Invocation for Multilingual ASR: A Cost-effective Approach Adapting to Speech Recognition Difficulty
Hongfei Xue, Yufeng Tang, Jun Zhang +2
Although multilingual automatic speech recognition (ASR) systems have significantly advanced, enabling a single model to handle multiple languages, inherent linguistic differences…
Enhancing Non-Core Language Instruction-Following in Speech LLMs via Semi-Implicit Cross-Lingual CoT Reasoning
Hongfei Xue, Yufeng Tang, Hexin Liu +3
Large language models have been extended to the speech domain, leading to the development of speech large language models (SLLMs). While existing SLLMs demonstrate strong performan…