43 citations · 184 across the 165 of their papers we have counts for
181 papers
XTurnix: Large-Scale Self-Supervised Turn Control through Two-State Binary Decisions
Zhanxun Liu, Yifan Duan, Hengtao Wu +7
General turn-taking behavior in real-time dialogue systems requires deciding whether to keep listening or start responding while listening, and whether to continue or stop while sp…
Pruned CTC for Memory-Efficient Large-Vocabulary ASR Training
Yifan Yang, Xiaoyu Yang, Zengrui Jin +10
Connectionist temporal classification (CTC) naturally supports offline and streaming speech recognition with utterance-level supervision, but conventional implementations materiali…
OmniVChat: Synthesizing, Benchmarking, and Training for Native Audio-Visual Dialogue
Haolin He, Yunfei Chu, Qi Chen +15
We define OmniVChat (Omni Video Chat) as the task of native audio-visual dialogue between a user and an omni model. In OmniVChat, omni models directly and simultaneously receive au…
Omni Demand Understanding: A Benchmark for Contextual User-Intent Inference in Multimodal Interaction
Qi Chen, Yunfei Chu, Haolin He +15
Natural audio-visual interaction is emerging as an important interface for AI assistants, allowing users to communicate through speech and vision rather than carefully composed tex…
VibeVoice-ASR-Streaming Technical Report
Yujie Tu, Zhiliang Peng, Jianwei Yu +11
Traditional speaker-attributed ASR systems treated ASR and speaker diarization as two separate tasks. Recently, end-to-end models such as VibeVoice-ASR have unified the two tasks w…
AuK Technical Report: An Open-Source Foundational Model for Speech Generation and Editing
Ziyang Ma, Zhikang Niu, Wenming Tu +30
We introduce AuK, an open-source foundational model that unifies speech generation and editing through a common interface of natural-language instructions and audio context. To sup…
The SLT 2026 SmartGlasses Challenge: Benchmarking Egocentric Multi-Talker Speech Recognition and Understanding with Audio-Language Models
Dehui Gao, Zhixian Zhao, Zhennan Lin +14
Recent advances in large language models (LLMs) and multimodal LLMs (MLLMs) have created new opportunities for wearable speech interfaces, with smart glasses providing an egocentri…
Efficient Chain-of-Modality Reasoning via Progressive Compression for Spoken Language Models
Pengchao Feng, Chao-Hong Tan, Qian Chen +3
Spoken language models (SLMs) enable natural human-computer interaction, but their reasoning ability still lags behind that of text-based large language models, especially on spoke…
Spoken Function Calling: A New Perspective on Spoken Language Understanding for Large Audio Language Models
Yuezhang Peng, Yuxin Liu, Changfeng Gao +4
Spoken Language Understanding (SLU) is the core component of task-oriented dialogue systems and a pivotal link in achieving seamless human-agent interaction. While traditional SLU…
GROW: Group-Relative Advantage-Weighted On-Policy Reinforcement Learning of Autoregressive-Diffusion Text-to-Speech model
Guanrou Yang, Tian Tan, Qian Chen +8
Reinforcement learning for flow-matching text-to-speech is complicated by deterministic ODE sampling: trajectory-level policy-gradient methods typically convert the ODE into an SDE…
OmniVAE: An Audio-Video VAE with Cross-Modal Alignment for Joint Generation
Jun Zhan, Chen Yang, Yitian Gong +23
Recent generative models are moving beyond silent video or standalone audio synthesis toward the joint generation of synchronized audio and video. Despite this progress, jointly ge…
AgenticASR: Refining Speech Recognition in Real-World Scenarios via an Agentic Approach
Zixuan Jiang, Binghao Qiang, Jiaying Chi +3
Automatic speech recognition (ASR) has achieved substantial gains in transcription accuracy, yet verbatim transcription does not necessarily produce readily usable text. It retains…
Summary of DCASE 2026 Task 5: Audio-Dependent Question Answering
Haolin He, Renhe Sun, Zheqi Dai +16
DCASE~2026 Task~5 introduces Audio-Dependent Question Answering (ADQA), which tests whether large audio-language models answer from the audio rather than from textual priors. An Au…
GigaSpeechBench: A Real-World Multilingual Speech-to-Text Benchmark
Yujie Tu, Yifan Yang, Tianrui Wang +35
While modern ASR systems achieve low error rates on high-resource benchmarks, such performance often overestimates real-world robustness. Existing evaluations address challenges in…
X-Translator: A Real-Time Multilingual Speaker-Aware Speech-to-Speech Translation System
Yuxiang Zhao, Yichi Zhang, Yanjie An +10
Real-time speech-to-speech translation (S2ST) systems must balance translation quality, latency, speech naturalness, and speaker consistency. Publicly documented S2ST systems have…
HoliDubber: Holistic Video Dubbing for Complex Acoustic Scenes via Text-Guided Audio Synthesis
Wenhao Guan, Yifan Duan, Junxi Liu +6
Video dubbing is a cornerstone of multimedia content creation, aiming to synthesize synchronized acoustic sequences for visual streams. While Text-to-Speech (TTS) and Text-to-Audio…
Speech Meets ELF: Audio Conditional Continuous-Target Diffusion for Speech Recognition and Translation
Xuanchen Li, Tianrui Wang, Yuheng Lu +11
Speech-to-text (S2T) systems for recognition (ASR) and translation (S2TT) typically generate discrete text tokens. In contrast, continuous-target language modelling performs genera…
Audio-Oscar: A Multi-Agent System for Complex Audio Scene Generation, Orchestration, and Refinement
Yifan Duan, Qixiang Xu, Hengtao Wu +6
In recent years, audio generation has made significant progress in tasks such as text-to-speech (TTS), text-to-audio (TTA) and text-to-music (TTM). However, generating long-form an…
MMAE: A Massive Multitask Audio Editing Benchmark
Ziyang Ma, Ruiqi Yan, Ruiyang Xu +35
We introduce MMAE, a Massive Multitask Audio Editing benchmark, serving as the first comprehensive evaluation testbed designed for general-purpose instruction-based audio editing.…
Read What You Hear: Reference-Free Hypotheses Evaluation with Acoustic Discrepancy
Zhihan Li, Hankun Wang, Yiwei Guo +3
Automatic speech recognition systems commonly rely on reference transcriptions for evaluation, while reference-free approaches often depend on internal confidence estimation or aux…
WavTTS: Towards High-Quality Zero-Shot TTS via Direct Raw Waveform Modeling
Wenxi Chen, Dongya Jia, Yushen Chen +11
Recently, diffusion models operating on VAE latents or mel-spectrograms have become the dominant paradigm for zero-shot TTS. Although these compressed representations improve gener…
AnyAudio-Judge: A Dynamic Rubric-Based Benchmark and Evaluator for Audio Instruction Following
Haitao Li, Tian Tan, Yuguang Yang +2
The rapid advancement of instruction-guided audio generation has highlighted the critical need for robust alignment evaluation. Current automated evaluation methods heavily rely on…
AudioDER: A Deduplication-Enhanced Reasoning Dataset for Post-Training Large Audio-Language Models
Hui Geng, Yi Su, Zijian Gao +5
Recent advances in pretrained large audio-language models (LALMs) have demonstrated strong capabilities across speech, sound, and music. To adapt these models to downstream tasks w…
VISA: A Visual Information Strengthened Audio-Reasoning System for the Interspeech 2026 ARC Agent Track
Wenming Tu, Jian Gao, Yanru Huo +9
Audio reasoning requires multi-step, evidence-grounded inference over temporally dynamic and acoustically mixed signals, exceeding conventional perception tasks such as ASR or capt…