2 citations · 5 across the 18 of their papers we have counts for
18 papers
The SLT 2026 SmartGlasses Challenge: Benchmarking Egocentric Multi-Talker Speech Recognition and Understanding with Audio-Language Models
Dehui Gao, Zhixian Zhao, Zhennan Lin +14
Recent advances in large language models (LLMs) and multimodal LLMs (MLLMs) have created new opportunities for wearable speech interfaces, with smart glasses providing an egocentri…
Beyond Semantic Dominance: Cognitive Affective Reasoning and Empathetic Response Alignment in Audio Language Models
Zhixian Zhao, Shuiyuan Wang, Wenjie Tian +3
While Audio Language Models (ALMs) demonstrate strong semantic understanding, they struggle with complex affective interactions. Specifically, textual semantic dominance often over…
S2Accompanist: A Semantic-Aware and Structure-Guided Diffusion Model for Music Accompaniment Generation
Huakang Chen, Wenkai Cheng, Guobin Ma +7
High-fidelity text-to-music generation typically relies on massive proprietary datasets and immense computational resources. Existing models often struggle to generate coherent pur…
Towards Fine-Grained Multi-Dimensional Speech Understanding: Data Pipeline, Benchmark, and Model
Guojian Li, Zhixian Zhao, Zhennan Lin +9
While speech Large Language Models (LLMs) excel at conventional tasks like basic speech recognition, they lack fine-grained, multi-dimensional perception. This deficiency is eviden…
Full-Duplex Interaction in Spoken Dialogue Systems: A Comprehensive Study from the ICASSP 2026 HumDial Challenge
Chengyou Wang, Hongfei Xue, Guojian Li +6
Full-duplex interaction, where speakers and listeners converse simultaneously, is a key element of human communication often missing from traditional spoken dialogue systems. These…
HumDial-EIBench: A Human-Recorded Multi-Turn Emotional Intelligence Benchmark for Audio Language Models
Shuiyuan Wang, Zhixian Zhao, Hongfei Xue +5
Evaluating the emotional intelligence (EI) of audio language models (ALMs) is critical. However, existing benchmarks mostly rely on synthesized speech, are limited to single-turn i…
Seeing the Context: Rich Visual Context-Aware Speech Recognition via Multimodal Reasoning
Wenjie Tian, Mingchen Shao, Bingshen Mu +8
Audio-visual speech recognition (AVSR) is an extension of ASR that incorporates visual signals. Current AVSR approaches primarily focus on lip motion, largely overlooking rich cont…
EmoOmni: Bridging Emotional Understanding and Expression in Omni-Modal LLMs
Wenjie Tian, Zhixian Zhao, Jingbin Hu +4
The evolution of Omni-Modal Large Language Models~(Omni-LLMs) has revolutionized human--computer interaction, enabling unified audio-visual perception and speech response. However,…
Integrating Fine-Grained Audio-Visual Evidence for Robust Multimodal Emotion Reasoning
Zhixian Zhao, Wenjie Tian, Lei Xie
Multimodal emotion analysis is shifting from static classification to generative reasoning. Beyond simple label prediction, robust affective reasoning must synthesize fine-grained…
The ICASSP 2026 HumDial Challenge: Benchmarking Human-like Spoken Dialogue Systems in the LLM Era
Zhixian Zhao, Shuiyuan Wang, Guojian Li +12
Driven by the rapid advancement of Large Language Models (LLMs), particularly Audio-LLMs and Omni-models, spoken dialogue systems have evolved significantly, progressively narrowin…
dLLM-ASR: A Faster Diffusion LLM-based Framework for Speech Recognition
Wenjie Tian, Bingshen Mu, Guobin Ma +3
Automatic speech recognition (ASR) systems based on large language models (LLMs) achieve superior performance by leveraging pretrained LLMs as decoders, but their token-by-token ge…
Serial-Parallel Dual-Path Architecture for Speaking Style Recognition
Guojian Li, Qijie Shao, Zhixian Zhao +3
Speaking Style Recognition (SSR) identifies a speaker's speaking style characteristics from speech. Existing style recognition approaches primarily rely on linguistic information,…
OSUM-EChat: Enhancing End-to-End Empathetic Spoken Chatbot via Understanding-Driven Spoken Dialogue
Xuelong Geng, Qijie Shao, Hongfei Xue +20
Empathy is crucial in enabling natural interactions within spoken dialogue systems, allowing machines to recognize and respond appropriately to paralinguistic cues such as age, gen…
DualDub: Video-to-Soundtrack Generation via Joint Speech and Background Audio Synthesis
Wenjie Tian, Xinfa Zhu, Haohe Liu +6
While recent video-to-audio (V2A) models can generate realistic background audio from visual input, they largely overlook speech, an essential part of many video soundtracks. This…
Spark-TTS: An Efficient LLM-Based Text-to-Speech Model with Single-Stream Decoupled Speech Tokens
Xinsheng Wang, Mingqi Jiang, Ziyang Ma +22
Recent advancements in large language models (LLMs) have driven significant progress in zero-shot text-to-speech (TTS) synthesis. However, existing foundation models rely on multi-…
OSUM: Advancing Open Speech Understanding Models with Limited Resources in Academia
Xuelong Geng, Kun Wei, Qijie Shao +18
Large Language Models (LLMs) have made significant progress in various downstream tasks, inspiring the development of Speech Understanding Language Models (SULMs) to enable compreh…
Steering Language Model to Stable Speech Emotion Recognition via Contextual Perception and Chain of Thought
Zhixian Zhao, Xinfa Zhu, Xinsheng Wang +4
Large-scale audio language models (ALMs), such as Qwen2-Audio, are capable of comprehending diverse audio signal, performing audio analysis and generating textual responses. Howeve…
Improving Multimodal Emotion Recognition by Leveraging Acoustic Adaptation and Visual Alignment
Zhixian Zhao, Haifeng Chen, Xi Li +2
Multimodal Emotion Recognition (MER) aims to automatically identify and understand human emotional states by integrating information from various modalities. However, the scarcity…