19 papers
Gradient-Based Speech-to-Text Alignment for Any ASR Model: From CTC to Speech LLMs
Albert Zeyer, Ralf Schlüter, Hermann Ney
Speech-to-text alignment means finding the temporal boundaries of each word in the audio. Some models provide such an alignment directly and others do not. Connectionist temporal c…
Revisiting the Relation Between Language Model Perplexity and ASR Word Error Rate for Modern End-to-End Speech Recognition
Mohammad Zeineldeen, Albert Zeyer, Haoran Zhang +3
Language model (LM) perplexity (PPL) has historically been used as a proxy for automatic speech recognition (ASR) word error rate (WER), with prior work reporting an approximately…
Enhancing BEST-RQ Pseudo-Label Quality through Online Refinement for Automatic Speech Recognition
Jingjing Xu, Zijian Yang, Mohammad Zeineldeen +3
BEST-RQ is a simple and effective self-supervised training method for speech representation learning that performs well on automatic speech recognition (ASR) tasks. It generates ps…
Text-Utilization for Encoder-dominated Speech Recognition Models
Albert Zeyer, Tim Posielek, Ralf Schlüter +1
This paper investigates efficient methods for utilizing text-only data to improve speech recognition, focusing on encoder-dominated models that facilitate faster recognition. We pr…
Diffusion Language Models for Speech Recognition
Davyd Naveriani, Albert Zeyer, Ralf Schlüter +1
Diffusion language models have recently emerged as a leading alternative to standard language models, due to their ability for bidirectional attention and parallel text generation.…
Sequence-Level Unsupervised Training in Speech Recognition: A Theoretical Study
Zijian Yang, Jörg Barkoczi, Ralf Schlüter +1
Unsupervised speech recognition is a task of training a speech recognition model with unpaired data. To determine when and how unsupervised speech recognition can succeed, and how…
LLMs and Speech: Integration vs. Combination
Robin Schmitt, Albert Zeyer, Mohammad Zeineldeen +2
In this work, we study different approaches to utilize large language models (LLMs) for automatic speech recognition (ASR). Specifically, we compare the tight integration of an aco…
Unified Learnable 2D Convolutional Feature Extraction for ASR
Peter Vieting, Benedikt Hilmes, Ralf Schlüter +1
Neural front-ends represent a promising approach to feature extraction for automatic speech recognition (ASR) systems as they enable to learn specifically tailored features for dif…
A Comparative Analysis on ASR System Combination for Attention, CTC, Factored Hybrid, and Transducer Models
Noureldin Bayoumi, Robin Schmitt, Tina Raissi +3
Combination approaches for speech recognition (ASR) systems cover structured sentence-level or word-based merging techniques as well as combination of model scores during beam sear…
Dynamic Acoustic Model Architecture Optimization in Training for ASR
Jingjing Xu, Zijian Yang, Albert Zeyer +3
Architecture design is inherently complex. Existing approaches rely on either handcrafted rules, which demand extensive empirical expertise, or automated methods like neural archit…
Label-Context-Dependent Internal Language Model Estimation for CTC
Zijian Yang, Minh-Nghia Phan, Ralf Schlüter +1
Although connectionist temporal classification (CTC) has the label context independence assumption, it can still implicitly learn a context-dependent internal language model (ILM)…
Regularizing Learnable Feature Extraction for Automatic Speech Recognition
Peter Vieting, Maximilian Kannen, Benedikt Hilmes +2
Neural front-ends are an appealing alternative to traditional, fixed feature extraction pipelines for automatic speech recognition (ASR) systems since they can be directly trained…
Classification Error Bound for Low Bayes Error Conditions in Machine Learning
Zijian Yang, Vahe Eminyan, Ralf Schlüter +1
In statistical classification and machine learning, classification error is an important performance measure, which is minimized by the Bayes decision rule. In practice, the unknow…
Right Label Context in End-to-End Training of Time-Synchronous ASR Models
Tina Raissi, Ralf Schlüter, Hermann Ney
Current time-synchronous sequence-to-sequence automatic speech recognition (ASR) models are trained by using sequence level cross-entropy that sums over all alignments. Due to the…
The Conformer Encoder May Reverse the Time Dimension
Robin Schmitt, Albert Zeyer, Mohammad Zeineldeen +2
We sometimes observe monotonically decreasing cross-attention weights in our Conformer-based global attention-based encoder-decoder (AED) models, Further investigation shows that t…
Refined Statistical Bounds for Classification Error Mismatches with Constrained Bayes Error
Zijian Yang, Vahe Eminyan, Ralf Schlüter +1
In statistical classification/multiple hypothesis testing and machine learning, a model distribution estimated from the training data is usually applied to replace the unknown true…
Investigating the Effect of Label Topology and Training Criterion on ASR Performance and Alignment Quality
Tina Raissi, Christoph Lüscher, Simon Berger +2
The ongoing research scenario for automatic speech recognition (ASR) envisions a clear division between end-to-end approaches and classic modular systems. Even though a high-level…
Investigating the Effect of Language Models in Sequence Discriminative Training for Neural Transducers
Zijian Yang, Wei Zhou, Ralf Schlüter +1
In this work, we investigate the effect of language models (LMs) with different context lengths and label units (phoneme vs. word) used in sequence discriminative training for phon…
End-to-End Training of a Neural HMM with Label and Transition Probabilities
Daniel Mann, Tina Raissi, Wilfried Michel +2
We investigate a novel modeling approach for end-to-end neural network training using hidden Markov models (HMM) where the transition probabilities between hidden states are modele…