21 citations · 154 across the 60 of their papers we have counts for
39 papers · 1 filter
Phoneme-First Prediction for LLM-Based Speech Recognition
Jakob Poncelet, Hugo Van hamme
Recent research has explored integrating Large Language Models (LLMs) with speech encoders to create speech-augmented LLMs capable of contextualized speech recognition. The main ch…
Speech Encoder Fusion for LLM-based Automatic Speech Recognition
Jakob Poncelet, Hugo Van hamme
Speech-aware large language models (LLMs) can incorporate speech through pre-trained acoustic encoders that project speech features into the LLM embedding space. While the choice o…
Towards Deep Contextual Reasoning from Broad Descriptions for ASR with Speech-LLM via Metadata-Driven Reasoning Chains
Jakob Poncelet, Hugo Van hamme
Speech recognition often fails on rare, domain-specific terms and context-related named entities. Existing contextualization techniques typically bias decoding with keywords or phr…
Parameter-Efficient Continual Learning for Automatic Speech Recognition
Steven Vander Eeckt, Hugo Van hamme
Speech foundation models enable strong general-purpose ASR and are attractive for downstream adaptation. However, their size and the catastrophic forgetting induced by sequential f…
Inverse-Hessian Regularization for Continual Learning in ASR
Steven Vander Eeckt, Hugo Van hamme
Catastrophic forgetting remains a major challenge for continual learning (CL) in automatic speech recognition (ASR), where models must adapt to new domains without losing performan…
Graph Connectionist Temporal Classification for Phoneme Recognition
Henry Grafé, Hugo Van hamme
Automatic Phoneme Recognition (APR) systems are often trained using pseudo phoneme-level annotations generated from text through Grapheme-to-Phoneme (G2P) systems. These G2P system…
Leveraging Broadcast Media Subtitle Transcripts for Automatic Speech Recognition and Subtitling
Jakob Poncelet, Hugo Van hamme
The recent advancement of speech recognition technology has been driven by large-scale datasets and attention-based architectures, but many challenges still remain, especially for…
Disentangled-Transformer: An Explainable End-to-End Automatic Speech Recognition Model with Speech Content-Context Separation
Pu Wang, Hugo Van hamme
End-to-end transformer-based automatic speech recognition (ASR) systems often capture multiple speech traits in their learned representations that are highly entangled, leading to…
Efficient Extraction of Noise-Robust Discrete Units from Self-Supervised Speech Models
Jakob Poncelet, Yujun Wang, Hugo Van hamme
Continuous speech can be converted into a discrete sequence by deriving discrete units from the hidden features of self-supervised learned (SSL) speech models. Although SSL models…
Unsupervised Online Continual Learning for Automatic Speech Recognition
Steven Vander Eeckt, Hugo Van hamme
Adapting Automatic Speech Recognition (ASR) models to new domains leads to Catastrophic Forgetting (CF) of previously learned information. This paper addresses CF in the challengin…
Unsupervised Accent Adaptation Through Masked Language Model Correction Of Discrete Self-Supervised Speech Units
Jakob Poncelet, Hugo Van hamme
Self-supervised pre-trained speech models have strongly improved speech recognition, yet they are still sensitive to domain shifts and accented or atypical speech. Many of these mo…
Analysis of XLS-R for Speech Quality Assessment
Bastiaan Tamm, Rik Vandenberghe, Hugo Van hamme
In online conferencing applications, estimating the perceived quality of an audio signal is crucial to ensure high quality of experience for the end user. The most reliable way to…
Rehearsal-Free Online Continual Learning for Automatic Speech Recognition
Steven Vander Eeckt, Hugo Van hamme
Fine-tuning an Automatic Speech Recognition (ASR) model to new domains results in degradation on original domains, referred to as Catastrophic Forgetting (CF). Continual Learning (…
Parameter-efficient Dysarthric Speech Recognition Using Adapter Fusion and Householder Transformation
Jinzi Qi, Hugo Van hamme
In dysarthric speech recognition, data scarcity and the vast diversity between dysarthric speakers pose significant challenges. While finetuning has been a popular solution, it can…
Cross-Lingual Transfer Learning for Alzheimer's Detection From Spontaneous Speech
Bastiaan Tamm, Rik Vandenberghe, Hugo Van hamme
Alzheimer's disease (AD) is a progressive neurodegenerative disease most often associated with memory deficits and cognitive decline. With the aging population, there has been much…
Impact of visual assistance for automated audio captioning
Wim Boes, Hugo Van hamme
We study the impact of visual assistance for automated audio captioning. Utilizing multi-encoder transformer architectures, which have previously been employed to introduce vision-…
Weak-Supervised Dysarthria-invariant Features for Spoken Language Understanding using an FHVAE and Adversarial Training
Jinzi Qi, Hugo Van hamme
The scarcity of training data and the large speaker variation in dysarthric speech lead to poor accuracy and poor speaker generalization of spoken language understanding systems fo…
Multi-Source Transformer Architectures for Audiovisual Scene Classification
Wim Boes, Hugo Van hamme
In this technical report, the systems we submitted for subtask 1B of the DCASE 2021 challenge, regarding audiovisual scene classification, are described in detail. They are essenti…
Optimizing Temporal Resolution Of Convolutional Recurrent Neural Networks For Sound Event Detection
Wim Boes, Hugo Van hamme
In this technical report, the systems we submitted for subtask 4 of the DCASE 2021 challenge, regarding sound event detection, are described in detail. These models are closely rel…
Learning to Jointly Transcribe and Subtitle for End-to-End Spontaneous Speech Recognition
Jakob Poncelet, Hugo Van hamme
TV subtitles are a rich source of transcriptions of many types of speech, ranging from read speech in news reports to conversational and spontaneous speech in talk shows and soaps.…
Weight Averaging: A Simple Yet Effective Method to Overcome Catastrophic Forgetting in Automatic Speech Recognition
Steven Vander Eeckt, Hugo Van hamme
Adapting a trained Automatic Speech Recognition (ASR) model to new tasks results in catastrophic forgetting of old tasks, limiting the model's ability to learn continually and to b…
Pre-trained Speech Representations as Feature Extractors for Speech Quality Assessment in Online Conferencing Applications
Bastiaan Tamm, Helena Balabin, Rik Vandenberghe +1
Speech quality in online conferencing applications is typically assessed through human judgements in the form of the mean opinion score (MOS) metric. Since such a labor-intensive a…
Impact of temporal resolution on convolutional recurrent networks for audio tagging and sound event detection
Wim Boes, Hugo Van hamme
Many state-of-the-art systems for audio tagging and sound event detection employ convolutional recurrent neural architectures. Typically, they are trained in a mean teacher setting…
Relating the fundamental frequency of speech with EEG using a dilated convolutional network
Corentin Puffay, Jana Van Canneyt, Jonas Vanthornhout +2
To investigate how speech is processed in the brain, we can model the relation between features of a natural speech signal and the corresponding recorded electroencephalogram (EEG)…