67 citations · 344 across the 35 of their papers we have counts for
28 papers · 1 filter
MiDashengLM-Spatial: Unifying General Audio Understanding and Spatial Awareness
Jinbo Hu, Hang Su, Lichun Fan +10
Large audio-language models (LALMs) have achieved strong performance in general audio understanding, yet most are designed for monaural input and discard the inter-channel cues ess…
Dasheng AudioGen: A Unified Model for Generating Coherent Audio Scenes from Text
Jiahao Mei, Heinrich Dinkel, Yadong Niu +7
Audio generation has long been fragmented, with speech, music, and sound effects produced by domain-specific models that fail to jointly generate coherent audio scenes from a singl…
DashengTokenizer: One layer is enough for unified audio understanding and generation
Heinrich Dinkel, Xingwei Sun, Gang Li +8
This paper introduces DashengTokenizer, a continuous audio tokenizer engineered for joint use in both understanding and generation tasks. Unlike conventional approaches, which trai…
The Interspeech 2026 Audio Encoder Capability Challenge for Large Audio Language Models
Heinrich Dinkel, Jiahao Zhou, Guanbo Wang +8
This paper presents the Interspeech 2026 Audio Encoder Capability Challenge, a benchmark specifically designed to evaluate and advance the performance of pre-trained audio encoders…
MiDashengLM: Efficient Audio Understanding with General Audio Captions
Heinrich Dinkel, Gang Li, Jizhong Liu +7
Current approaches for large audio language models (LALMs) often rely on closed data sources or proprietary models, limiting their generalization and accessibility. This paper intr…
GLAP: General contrastive audio-text pretraining across domains and languages
Heinrich Dinkel, Zhiyong Yan, Tianzi Wang +7
Contrastive Language Audio Pretraining (CLAP) is a widely-used method to bridge the gap between audio and text domains. Current CLAP methods enable sound and music retrieval in Eng…
X-ARES: A Comprehensive Framework for Assessing Audio Encoder Performance
Junbo Zhang, Heinrich Dinkel, Yadong Niu +4
We introduces X-ARES (eXtensive Audio Representation and Evaluation Suite), a novel open-source benchmark designed to systematically assess audio encoder performance across diverse…
Reinforcement Learning Outperforms Supervised Fine-Tuning: A Case Study on Audio Question Answering
Gang Li, Jizhong Liu, Heinrich Dinkel +3
Recently, reinforcement learning (RL) has been shown to greatly enhance the reasoning capabilities of large language models (LLMs), and RL-based approaches have been progressively…
The ICME 2025 Audio Encoder Capability Challenge
Junbo Zhang, Heinrich Dinkel, Qiong Song +8
This challenge aims to evaluate the capabilities of audio encoders, especially in the context of multi-task learning and real-world applications. Participants are invited to submit…
Enhancing Automated Audio Captioning via Large Language Models with Optimized Audio Encoding
Jizhong Liu, Gang Li, Junbo Zhang +5
Automated audio captioning (AAC) is an audio-to-text task to describe audio contents in natural language. Recently, the advancements in large language models (LLMs), with improveme…
Bridging Language Gaps in Audio-Text Retrieval
Zhiyong Yan, Heinrich Dinkel, Yongqing Wang +4
Audio-text retrieval is a challenging task, requiring the search for an audio clip or a text caption within a database. The predominant focus of existing research on English descri…
Scaling up masked audio encoder learning for general audio classification
Heinrich Dinkel, Zhiyong Yan, Yongqing Wang +3
Despite progress in audio classification, a generalization gap remains between speech and other sound domains, such as environmental sounds and music. Models trained for speech tas…
CED: Consistent ensemble distillation for audio tagging
Heinrich Dinkel, Yongqing Wang, Zhiyong Yan +2
Augmentation and knowledge distillation (KD) are well-established techniques employed in audio classification tasks, aimed at enhancing performance and reducing model sizes on the…
Focus on the Sound around You: Monaural Target Speaker Extraction via Distance and Speaker Information
Jiuxin Lin, Peng Wang, Heinrich Dinkel +6
Previously, Target Speaker Extraction (TSE) has yielded outstanding performance in certain application scenarios for speech enhancement and source separation. However, obtaining au…
Understanding temporally weakly supervised training: A case study for keyword spotting
Heinrich Dinkel, Weiji Zhuang, Zhiyong Yan +3
The currently most prominent algorithm to train keyword spotting (KWS) models with deep neural networks (DNNs) requires strong supervision i.e., precise knowledge of the spoken key…
Streaming Audio Transformers for Online Audio Tagging
Heinrich Dinkel, Zhiyong Yan, Yongqing Wang +3
Transformers have emerged as a prominent model framework for audio tagging (AT), boasting state-of-the-art (SOTA) performance on the widely-used Audioset dataset. However, their im…
Unified Keyword Spotting and Audio Tagging on Mobile Devices with Transformers
Heinrich Dinkel, Yongqing Wang, Zhiyong Yan +2
Keyword spotting (KWS) is a core human-machine-interaction front-end task for most modern intelligent assistants. Recently, a unified (UniKW-AT) framework has been proposed that ad…
An empirical study of weakly supervised audio tagging embeddings for general audio representations
Heinrich Dinkel, Zhiyong Yan, Yongqing Wang +2
We study the usability of pre-trained weakly supervised audio tagging (AT) models as feature extractors for general audio representations. We mainly analyze the feasibility of tran…
UniKW-AT: Unified Keyword Spotting and Audio Tagging
Heinrich Dinkel, Yongqing Wang, Zhiyong Yan +2
Within the audio research community and the industry, keyword spotting (KWS) and audio tagging (AT) are seen as two distinct tasks and research fields. However, from a technical po…
Pseudo strong labels for large scale weakly supervised audio tagging
Heinrich Dinkel, Zhiyong Yan, Yongqing Wang +2
Large-scale audio tagging datasets inevitably contain imperfect labels, such as clip-wise annotated (temporally weak) tags with no exact on- and offsets, due to a high manual label…
Voice activity detection in the wild: A data-driven approach using teacher-student training
Heinrich Dinkel, Shuai Wang, Xuenan Xu +2
Voice activity detection is an essential pre-processing component for speech-related tasks such as automatic speech recognition (ASR). Traditional supervised VAD systems obtain fra…
Text-to-Audio Grounding: Building Correspondence Between Captions and Sound Events
Xuenan Xu, Heinrich Dinkel, Mengyue Wu +1
Automated Audio Captioning is a cross-modal task, generating natural language descriptions to summarize the audio clips' sound events. However, grounding the actual sound events in…
Investigating Local and Global Information for Automated Audio Captioning with Transfer Learning
Xuenan Xu, Heinrich Dinkel, Mengyue Wu +2
Automated audio captioning (AAC) aims at generating summarizing descriptions for audio clips. Multitudinous concepts are described in an audio caption, ranging from local informati…
Towards duration robust weakly supervised sound event detection
Heinrich Dinkel, Mengyue Wu, Kai Yu
Sound event detection (SED) is the task of tagging the absence or presence of audio events and their corresponding interval within a given audio clip. While SED can be done using s…