67 citations · 344 across the 35 of their papers we have counts for
37 papers
MiDashengLM-Spatial: Unifying General Audio Understanding and Spatial Awareness
Jinbo Hu, Hang Su, Lichun Fan +10
Large audio-language models (LALMs) have achieved strong performance in general audio understanding, yet most are designed for monaural input and discard the inter-channel cues ess…
MiDashengLM-Gen: Unified Audio Scene Generation via LLM-Driven Autoregressive Flow Matching
Xingwei Sun, Heinrich Dinkel, Gang Li +7
Generating coherent audio scenes that simultaneously blend speech, music, and sound effects remains a significant challenge. Current approaches typically rely on a disjointed pipel…
Dasheng AudioGen: A Unified Model for Generating Coherent Audio Scenes from Text
Jiahao Mei, Heinrich Dinkel, Yadong Niu +7
Audio generation has long been fragmented, with speech, music, and sound effects produced by domain-specific models that fail to jointly generate coherent audio scenes from a singl…
DashengTokenizer: One layer is enough for unified audio understanding and generation
Heinrich Dinkel, Xingwei Sun, Gang Li +8
This paper introduces DashengTokenizer, a continuous audio tokenizer engineered for joint use in both understanding and generation tasks. Unlike conventional approaches, which trai…
ACAVCaps: Enabling large-scale training for fine-grained and diverse audio understanding
Yadong Niu, Tianzi Wang, Heinrich Dinkel +6
General audio understanding is a fundamental goal for large audio-language models, with audio captioning serving as a cornerstone task for their development. However, progress in t…
The Interspeech 2026 Audio Encoder Capability Challenge for Large Audio Language Models
Heinrich Dinkel, Jiahao Zhou, Guanbo Wang +8
This paper presents the Interspeech 2026 Audio Encoder Capability Challenge, a benchmark specifically designed to evaluate and advance the performance of pre-trained audio encoders…
MiDashengLM: Efficient Audio Understanding with General Audio Captions
Heinrich Dinkel, Gang Li, Jizhong Liu +7
Current approaches for large audio language models (LALMs) often rely on closed data sources or proprietary models, limiting their generalization and accessibility. This paper intr…
MECAT: A Multi-Experts Constructed Benchmark for Fine-Grained Audio Understanding Tasks
Yadong Niu, Tianzi Wang, Heinrich Dinkel +7
While large audio-language models have advanced open-ended audio understanding, they still fall short of nuanced human-level comprehension. This gap persists largely because curren…
Efficient Speech Enhancement via Embeddings from Pre-trained Generative Audioencoders
Xingwei Sun, Heinrich Dinkel, Yadong Niu +3
Recent research has delved into speech enhancement (SE) approaches that leverage audio embeddings from pre-trained models, diverging from time-frequency masking or signal predictio…
GLAP: General contrastive audio-text pretraining across domains and languages
Heinrich Dinkel, Zhiyong Yan, Tianzi Wang +7
Contrastive Language Audio Pretraining (CLAP) is a widely-used method to bridge the gap between audio and text domains. Current CLAP methods enable sound and music retrieval in Eng…
X-ARES: A Comprehensive Framework for Assessing Audio Encoder Performance
Junbo Zhang, Heinrich Dinkel, Yadong Niu +4
We introduces X-ARES (eXtensive Audio Representation and Evaluation Suite), a novel open-source benchmark designed to systematically assess audio encoder performance across diverse…
Reinforcement Learning Outperforms Supervised Fine-Tuning: A Case Study on Audio Question Answering
Gang Li, Jizhong Liu, Heinrich Dinkel +3
Recently, reinforcement learning (RL) has been shown to greatly enhance the reasoning capabilities of large language models (LLMs), and RL-based approaches have been progressively…
The ICME 2025 Audio Encoder Capability Challenge
Junbo Zhang, Heinrich Dinkel, Qiong Song +8
This challenge aims to evaluate the capabilities of audio encoders, especially in the context of multi-task learning and real-world applications. Participants are invited to submit…
Enhancing Automated Audio Captioning via Large Language Models with Optimized Audio Encoding
Jizhong Liu, Gang Li, Junbo Zhang +5
Automated audio captioning (AAC) is an audio-to-text task to describe audio contents in natural language. Recently, the advancements in large language models (LLMs), with improveme…
Bridging Language Gaps in Audio-Text Retrieval
Zhiyong Yan, Heinrich Dinkel, Yongqing Wang +4
Audio-text retrieval is a challenging task, requiring the search for an audio clip or a text caption within a database. The predominant focus of existing research on English descri…
Scaling up masked audio encoder learning for general audio classification
Heinrich Dinkel, Zhiyong Yan, Yongqing Wang +3
Despite progress in audio classification, a generalization gap remains between speech and other sound domains, such as environmental sounds and music. Models trained for speech tas…
CED: Consistent ensemble distillation for audio tagging
Heinrich Dinkel, Yongqing Wang, Zhiyong Yan +2
Augmentation and knowledge distillation (KD) are well-established techniques employed in audio classification tasks, aimed at enhancing performance and reducing model sizes on the…
AV-SepFormer: Cross-Attention SepFormer for Audio-Visual Target Speaker Extraction
Jiuxin Lin, Xinyu Cai, Heinrich Dinkel +7
Visual information can serve as an effective cue for target speaker extraction (TSE) and is vital to improving extraction performance. In this paper, we propose AV-SepFormer, a Sep…
Focus on the Sound around You: Monaural Target Speaker Extraction via Distance and Speaker Information
Jiuxin Lin, Peng Wang, Heinrich Dinkel +6
Previously, Target Speaker Extraction (TSE) has yielded outstanding performance in certain application scenarios for speech enhancement and source separation. However, obtaining au…
Understanding temporally weakly supervised training: A case study for keyword spotting
Heinrich Dinkel, Weiji Zhuang, Zhiyong Yan +3
The currently most prominent algorithm to train keyword spotting (KWS) models with deep neural networks (DNNs) requires strong supervision i.e., precise knowledge of the spoken key…
Streaming Audio Transformers for Online Audio Tagging
Heinrich Dinkel, Zhiyong Yan, Yongqing Wang +3
Transformers have emerged as a prominent model framework for audio tagging (AT), boasting state-of-the-art (SOTA) performance on the widely-used Audioset dataset. However, their im…
Unified Keyword Spotting and Audio Tagging on Mobile Devices with Transformers
Heinrich Dinkel, Yongqing Wang, Zhiyong Yan +2
Keyword spotting (KWS) is a core human-machine-interaction front-end task for most modern intelligent assistants. Recently, a unified (UniKW-AT) framework has been proposed that ad…
An empirical study of weakly supervised audio tagging embeddings for general audio representations
Heinrich Dinkel, Zhiyong Yan, Yongqing Wang +2
We study the usability of pre-trained weakly supervised audio tagging (AT) models as feature extractors for general audio representations. We mainly analyze the feasibility of tran…
UniKW-AT: Unified Keyword Spotting and Audio Tagging
Heinrich Dinkel, Yongqing Wang, Zhiyong Yan +2
Within the audio research community and the industry, keyword spotting (KWS) and audio tagging (AT) are seen as two distinct tasks and research fields. However, from a technical po…