1 citations · 1 across the 7 of their papers we have counts for
11 papers
Preference-Aware Pareto Steering for Multi-Task Speech LLMs
Liuyuan Jiang, Vishal Sunder, Samuel Thomas +4
Speech large language models (LLMs) perform multiple tasks, including automatic speech recognition (ASR), timestamp prediction (TS), and speech translation (ST). Joint training of…
Design of the IBM Granite 5.0 TurboCTC ASR Model
Brian Kingsbury, George Saon, Masayuki Suzuki +5
We describe the architecture, training methodology and inference speedups of Granite 5.0 Turbo CTC, a 470 million parameter encoder-only model with an excellent speed-accuracy trad…
BiRQ: Bi-Level Self-Labeling Random Quantization for Self-Supervised Speech Recognition
Liuyuan Jiang, Xiaodong Cui, Brian Kingsbury +2
Speech is a rich signal, and labeled audio-text pairs are costly, making self-supervised learning essential for scalable representation learning. A core challenge in speech SSL is…
Heterogeneous Self-Supervised Acoustic Pre-Training with Local Constraints
Xiaodong Cui, A F M Saif, Brian Kingsbury +1
Self-supervised pre-training using unlabeled data is widely used in automatic speech recognition. In this paper, we propose a new self-supervised pre-training approach to dealing w…
Objective Soups: Multilingual Multi-Task Modeling for Speech Processing
A F M Saif, Lisha Chen, Xiaodong Cui +3
Training a single model for multilingual, multi-task speech processing (MSP) is severely hampered by conflicting objectives between tasks like speech recognition and translation. W…
Granite-speech: open-source speech-aware LLMs with strong English ASR capabilities
George Saon, Avihu Dekel, Alexander Brooks +21
Granite-speech LLMs are compact and efficient speech language models specifically designed for English ASR and automatic speech translation (AST). The models were trained by modali…
A Non-autoregressive Model for Joint STT and TTS
Vishal Sunder, Brian Kingsbury, George Saon +5
In this paper, we take a step towards jointly modeling automatic speech recognition (STT) and speech synthesis (TTS) in a fully non-autoregressive way. We develop a novel multimoda…
Bilevel Joint Unsupervised and Supervised Training for Automatic Speech Recognition
Xiaodong Cui, A F M Saif, Songtao Lu +4
In this paper, we propose a bilevel joint unsupervised and supervised training (BL-JUST) framework for automatic speech recognition. Compared to the conventional pre-training and f…
Exploring the limits of decoder-only models trained on public speech recognition corpora
Ankit Gupta, George Saon, Brian Kingsbury
The emergence of industrial-scale speech recognition (ASR) models such as Whisper and USM, trained on 1M hours of weakly labelled and 12M hours of audio only proprietary data respe…
Joint Unsupervised and Supervised Training for Automatic Speech Recognition via Bilevel Optimization
A F M Saif, Xiaodong Cui, Han Shen +3
In this paper, we present a novel bilevel optimization-based training approach to training acoustic models for automatic speech recognition (ASR) tasks that we term {bi-level joint…
Soft Random Sampling: A Theoretical and Empirical Analysis
Xiaodong Cui, Ashish Mittal, Songtao Lu +3
Soft random sampling (SRS) is a simple yet effective approach for efficient training of large-scale deep neural networks when dealing with massive data. SRS selects a subset unifor…