21 citations · 154 across the 60 of their papers we have counts for
65 papers
Phoneme-First Prediction for LLM-Based Speech Recognition
Jakob Poncelet, Hugo Van hamme
Recent research has explored integrating Large Language Models (LLMs) with speech encoders to create speech-augmented LLMs capable of contextualized speech recognition. The main ch…
Speech Encoder Fusion for LLM-based Automatic Speech Recognition
Jakob Poncelet, Hugo Van hamme
Speech-aware large language models (LLMs) can incorporate speech through pre-trained acoustic encoders that project speech features into the LLM embedding space. While the choice o…
Towards Deep Contextual Reasoning from Broad Descriptions for ASR with Speech-LLM via Metadata-Driven Reasoning Chains
Jakob Poncelet, Hugo Van hamme
Speech recognition often fails on rare, domain-specific terms and context-related named entities. Existing contextualization techniques typically bias decoding with keywords or phr…
Parameter-Efficient Continual Learning for Automatic Speech Recognition
Steven Vander Eeckt, Hugo Van hamme
Speech foundation models enable strong general-purpose ASR and are attractive for downstream adaptation. However, their size and the catastrophic forgetting induced by sequential f…
GLoRIA: Gated Low-Rank Interpretable Adaptation for Dialectal ASR
Pouya Mehralian, Melissa Farasyn, Anne Breitbarth +2
Automatic Speech Recognition (ASR) in dialect-heavy settings remains challenging due to strong regional variation and limited labeled data. We propose GLoRIA, a parameter-efficient…
Inverse-Hessian Regularization for Continual Learning in ASR
Steven Vander Eeckt, Hugo Van hamme
Catastrophic forgetting remains a major challenge for continual learning (CL) in automatic speech recognition (ASR), where models must adapt to new domains without losing performan…
SSVD-O: Parameter-Efficient Fine-Tuning with Structured SVD for Speech Recognition
Pu Wang, Shinji Watanabe, Hugo Van hamme
Parameter-efficient fine-tuning (PEFT) is a scalable approach for adapting large speech foundation models to new domains. While methods such as LoRA and its state-of-the-art varian…
Graph Connectionist Temporal Classification for Phoneme Recognition
Henry Grafé, Hugo Van hamme
Automatic Phoneme Recognition (APR) systems are often trained using pseudo phoneme-level annotations generated from text through Grapheme-to-Phoneme (G2P) systems. These G2P system…
SSVD: Structured SVD for Parameter-Efficient Fine-Tuning and Benchmarking under Domain Shift in ASR
Pu Wang, Shinji Watanabe, Hugo Van hamme
Parameter-efficient fine-tuning (PEFT) has emerged as a scalable solution for adapting large foundation models. While low-rank adaptation (LoRA) is widely used in speech applicatio…
Refining Transcripts With TV Subtitles by Prompt-Based Weakly Supervised Training of ASR
Xinnian Zhao, Hugo Van Hamme
This study proposes a novel approach to using TV subtitles within a weakly supervised (WS) Automatic Speech Recognition (ASR) framework. Although TV subtitles are readily available…
Benchmarking Training Paradigms, Dataset Composition, and Model Scaling for Child ASR in ESPnet
Anyu Ying, Natarajan Balaji Shankar, Chyi-Jiunn Lin +7
Despite advancements in ASR, child speech recognition remains challenging due to acoustic variability and limited annotated data. While fine-tuning adult ASR models on child speech…
Leveraging Broadcast Media Subtitle Transcripts for Automatic Speech Recognition and Subtitling
Jakob Poncelet, Hugo Van hamme
The recent advancement of speech recognition technology has been driven by large-scale datasets and attention-based architectures, but many challenges still remain, especially for…
Disentangled-Transformer: An Explainable End-to-End Automatic Speech Recognition Model with Speech Content-Context Separation
Pu Wang, Hugo Van hamme
End-to-end transformer-based automatic speech recognition (ASR) systems often capture multiple speech traits in their learned representations that are highly entangled, leading to…
Efficient Extraction of Noise-Robust Discrete Units from Self-Supervised Speech Models
Jakob Poncelet, Yujun Wang, Hugo Van hamme
Continuous speech can be converted into a discrete sequence by deriving discrete units from the hidden features of self-supervised learned (SSL) speech models. Although SSL models…
Automatic recognition and detection of aphasic natural speech
Mara Barberis, Pieter De Clercq, Bastiaan Tamm +2
Aphasia is a language disorder affecting one third of stroke patients. Current aphasia assessment does not consider natural speech due to the time consuming nature of manual transc…
Unsupervised Online Continual Learning for Automatic Speech Recognition
Steven Vander Eeckt, Hugo Van hamme
Adapting Automatic Speech Recognition (ASR) models to new domains leads to Catastrophic Forgetting (CF) of previously learned information. This paper addresses CF in the challengin…
MSNER: A Multilingual Speech Dataset for Named Entity Recognition
Quentin Meeus, Marie-Francine Moens, Hugo Van hamme
While extensively explored in text-based tasks, Named Entity Recognition (NER) remains largely neglected in spoken language understanding. Existing resources are limited to a singl…
Detecting Post-Stroke Aphasia Via Brain Responses to Speech in a Deep Learning Framework
Pieter De Clercq, Corentin Puffay, Jill Kries +4
Aphasia, a language disorder primarily caused by a stroke, is traditionally diagnosed using behavioral language tests. However, these tests are time-consuming, require manual inter…
Unsupervised Accent Adaptation Through Masked Language Model Correction Of Discrete Self-Supervised Speech Units
Jakob Poncelet, Hugo Van hamme
Self-supervised pre-trained speech models have strongly improved speech recognition, yet they are still sensitive to domain shifts and accented or atypical speech. Many of these mo…
Analysis of XLS-R for Speech Quality Assessment
Bastiaan Tamm, Rik Vandenberghe, Hugo Van hamme
In online conferencing applications, estimating the perceived quality of an audio signal is crucial to ensure high quality of experience for the end user. The most reliable way to…
Rehearsal-Free Online Continual Learning for Automatic Speech Recognition
Steven Vander Eeckt, Hugo Van hamme
Fine-tuning an Automatic Speech Recognition (ASR) model to new domains results in degradation on original domains, referred to as Catastrophic Forgetting (CF). Continual Learning (…
Parameter-efficient Dysarthric Speech Recognition Using Adapter Fusion and Householder Transformation
Jinzi Qi, Hugo Van hamme
In dysarthric speech recognition, data scarcity and the vast diversity between dysarthric speakers pose significant challenges. While finetuning has been a popular solution, it can…
Cross-Lingual Transfer Learning for Alzheimer's Detection From Spontaneous Speech
Bastiaan Tamm, Rik Vandenberghe, Hugo Van hamme
Alzheimer's disease (AD) is a progressive neurodegenerative disease most often associated with memory deficits and cognitive decline. With the aging population, there has been much…
CLASH: Contrastive learning through alignment shifting to extract stimulus information from EEG
Bernd Accou, Hugo Van hamme, Tom Francart
Stimulus-evoked EEG data has a notoriously low signal-to-noise ratio and high inter-subject variability. We propose a novel paradigm for the self-supervised extraction of stimulus-…