6 papers
ROAM-ASD: Robust Open-World Active Speaker Detection with Flexible Multimodal Fusion
Pu Wang, Hugo Van hamme
Active speaker detection (ASD) requires reliable association between visible faces and acoustic speech, yet existing systems often degrade under challenging domains or incomplete o…
LiveGesture Streamable Co-Speech Gesture Generation Model
Muhammad Usama Saleem, Mayur Jagdishbhai Patel, Ekkasit Pinyoanuntapong +6
We propose LiveGesture, the first fully streamable, speech-driven full-body gesture generation framework that operates with zero look-ahead and supports arbitrary sequence length.…
MAGE: Modality-Agnostic Music Generation and Target-Source Extraction
Muhammad Usama Saleem, Tejasvi Ravi, Tianyu Xu +4
Recent advances in multimodal audio generation have enabled music synthesis from text, visual cues, and other high-level conditions. However, most systems are designed for a single…
SSVD-O: Parameter-Efficient Fine-Tuning with Structured SVD for Speech Recognition
Pu Wang, Shinji Watanabe, Hugo Van hamme
Parameter-efficient fine-tuning (PEFT) is a scalable approach for adapting large speech foundation models to new domains. While methods such as LoRA and its state-of-the-art varian…
SSVD: Structured SVD for Parameter-Efficient Fine-Tuning and Benchmarking under Domain Shift in ASR
Pu Wang, Shinji Watanabe, Hugo Van hamme
Parameter-efficient fine-tuning (PEFT) has emerged as a scalable solution for adapting large foundation models. While low-rank adaptation (LoRA) is widely used in speech applicatio…
Benchmarking Training Paradigms, Dataset Composition, and Model Scaling for Child ASR in ESPnet
Anyu Ying, Natarajan Balaji Shankar, Chyi-Jiunn Lin +7
Despite advancements in ASR, child speech recognition remains challenging due to acoustic variability and limited annotated data. While fine-tuning adult ASR models on child speech…