12 papers
VoxMem: Benchmarking Multimodal Memory in Large Audio Language Models
Yang Xiao, Vidhyasaharan Sethu, Eun-Jung Holden +1
Spoken conversational systems must recover information from prior interactions (i.e., memory), yet relevant information in speech extends beyond what was said to who said it, how i…
QuaSR: Quality-Aware Sample Reweighting for Pacific Indigenous Speech Recognition
Yishun Li, Yang Xiao, Gongping Huang +3
Training automatic speech recognition (ASR) models for low-resource languages is challenging due to limited data and highly variable supervision quality. In particular, Pacific Ind…
RAIL: Rethinking Auditory Intelligence in Large Audio-Language Models with a CHC-Grounded Benchmark
Hongyu Jin, Siyi Wang, Yang Xiao +11
Humans process rich auditory environments through various cognitive capabilities such as audio perception, audio reasoning and memory. Despite recent progress in large audio-langua…
Rethinking Continual Learning for Speech and Audio: A Representation-Centric Taxonomy and Open Problems
Yang Xiao, Siyi Wang, Eun-Jung Holden +1
Speech and audio systems operate in inherently non-stationary environments, yet continual learning (CL) research in this domain, especially in the foundation model era, remains fra…
Why Can't They Remember? Uncovering Representation and Retrieval Bottlenecks in Multi-Turn Acoustic Memory
Yang Xiao, Siyi Wang, Han Yin +4
Large audio language models (LALMs) process both speech and environmental acoustic cues, yet struggle to retain non-speech information across multi-turn interactions. The performan…
Continual Adaptation for Pacific Indigenous Speech Recognition
Yang Xiao, Aso Mahmudi, Nick Thieberger +3
Speech foundation models struggle with low-resource Pacific Indigenous languages because of severe data scarcity. Furthermore, full fine-tuning risks catastrophic forgetting. To ad…
Activation Steering for Accent Adaptation in Large Audio Language Models
Jinuo Sun, Yang Xiao, Sung Kyun Chung +4
Accent variability remains a major source of errors in automatic speech recognition, yet most adaptation methods rely on parameter fine-tuning without understanding where accent in…
FiLoRA: Focus-and-Ignore LoRA for Controllable Feature Reliance
Hyunsuk Chung, Soyeon Caren Han, Seungyeon Ji +3
Multimodal foundation models integrate heterogeneous signals across modalities, yet it remains unclear whether their predictions can be controlled by explicitly modulating reliance…
Adapting Where It Matters: Depth-Aware Adaptation for Efficient Multilingual Speech Recognition in Low-Resource Languages
Yang Xiao, Eun-Jung Holden, Ting Dang
Recent speech foundation models excel at multilingual automatic speech recognition (ASR) for high-resource languages, but adapting them to low-resource languages remains challengin…
MUSEKG: A Knowledge Graph Over Museum Collections
Jinhao Li, Jianzhong Qi, Soyeon Caren Han +1
Digitisation in the cultural heritage sector has produced large but fragmented repositories of museum collection data, spanning structured catalogue records, images, and unstructur…
SPANER: Shared Prompt Aligner for Multimodal Semantic Representation
Thye Shan Ng, Caren Soyeon Han, Eun-Jung Holden
Recent advances in multimodal Parameter-Efficient Fine-Tuning (PEFT) have significantly improved performance on downstream tasks such as few-shot retrieval. However, most existing…
SymbioticRAG: Enhancing Document Intelligence Through Human-LLM Symbiotic Collaboration
Qiang Sun, Tingting Bi, Sirui Li +4
We present \textbf{SymbioticRAG}, a novel framework that fundamentally reimagines Retrieval-Augmented Generation~(RAG) systems by establishing a bidirectional learning relationship…