12 papers
HalluAudio: A Comprehensive Benchmark for Hallucination Detection in Large Audio-Language Models
Feiyu Zhao, Yiming Chen, Wenhuan Lu +3
Large Audio-Language Models (LALMs) have recently achieved strong performance across various audio-centric tasks. However, hallucination, where models generate responses that are s…
Whisper-MLA: Reducing GPU Memory Consumption of ASR Models based on MHA2MLA Conversion
Sen Zhang, Jianguo Wei, Wenhuan Lu +6
The Transformer-based Whisper model has achieved state-of-the-art performance in Automatic Speech Recognition (ASR). However, its Multi-Head Attention (MHA) mechanism results in si…
Listening for "You": Enhancing Speech Image Retrieval via Target Speaker Extraction
Wenhao Yang, Jianguo Wei, Wenhuan Lu +2
Image retrieval using spoken language cues has emerged as a promising direction in multimodal perception, yet leveraging speech in multi-speaker scenarios remains challenging. We p…
Audio-Visual Class-Incremental Learning for Fish Feeding intensity Assessment in Aquaculture
Meng Cui, Xianghu Yue, Xinyuan Qian +5
Fish Feeding Intensity Assessment (FFIA) is crucial in industrial aquaculture management. Recent multi-modal approaches have shown promise in improving FFIA robustness and efficien…
UniCodec: Unified Audio Codec with Single Domain-Adaptive Codebook
Yidi Jiang, Qian Chen, Shengpeng Ji +6
The emergence of audio language models is empowered by neural audio codecs, which establish critical mappings between continuous waveforms and discrete tokens compatible with langu…
PAL: Prompting Analytic Learning with Missing Modality for Multi-Modal Class-Incremental Learning
Xianghu Yue, Yiming Chen, Xueyi Zhang +5
Multi-modal class-incremental learning (MMCIL) seeks to leverage multi-modal data, such as audio-visual and image-text pairs, thereby enabling models to learn continuously across a…