93 citations · 198 across the 19 of their papers we have counts for
22 papers
LegoSLM: Connecting LLM with Speech Encoder using CTC Posteriors
Rao Ma, Tongzhou Chen, Kartik Audhkhasi +1
Recently, large-scale pre-trained speech encoders and Large Language Models (LLMs) have been released, which show state-of-the-art performance on a range of spoken language process…
STAB: Speech Tokenizer Assessment Benchmark
Shikhar Vashishth, Harman Singh, Shikhar Bharadwaj +6
Representing speech as discrete tokens provides a framework for transforming speech into a format that closely resembles text, thus enabling the use of speech as an input to the wi…
O-1: Self-training with Oracle and 1-best Hypothesis
Murali Karthick Baskar, Andrew Rosenberg, Bhuvana Ramabhadran +1
We introduce O-1, a new self-training objective to reduce training bias and unify training and evaluation metrics for speech recognition. O-1 is a faster variant of Expected Minimu…
Large-scale Language Model Rescoring on Long-form Data
Tongzhou Chen, Cyril Allauzen, Yinghui Huang +8
In this work, we study the impact of Large-scale Language Models (LLM) on Automated Speech Recognition (ASR) of YouTube videos, which we use as a source for long-form ASR. We demon…
Robust Knowledge Distillation from RNN-T Models With Noisy Training Labels Using Full-Sum Loss
Mohammad Zeineldeen, Kartik Audhkhasi, Murali Karthick Baskar +1
This work studies knowledge distillation (KD) and addresses its constraints for recurrent neural network transducer (RNN-T) models. In hard distillation, a teacher model transcribe…
Modular Hybrid Autoregressive Transducer
Zhong Meng, Tongzhou Chen, Rohit Prabhavalkar +10
Text-only adaptation of a transducer model remains challenging for end-to-end speech recognition since the transducer has no clearly separated acoustic model (AM), language model (…
Analysis of Self-Attention Head Diversity for Conformer-based Automatic Speech Recognition
Kartik Audhkhasi, Yinghui Huang, Bhuvana Ramabhadran +1
Attention layers are an integral part of modern end-to-end automatic speech recognition systems, for instance as part of the Transformer or Conformer architecture. Attention is typ…
Leveraging Unpaired Text Data for Training End-to-End Speech-to-Intent Systems
Yinghui Huang, Hong-Kwang Kuo, Samuel Thomas +5
Training an end-to-end (E2E) neural network speech-to-intent (S2I) system that directly extracts intents from speech requires large amounts of intent-labeled speech data, which is…
End-to-End Spoken Language Understanding Without Full Transcripts
Hong-Kwang J. Kuo, Zoltán Tüske, Samuel Thomas +7
An essential component of spoken language understanding (SLU) is slot filling: representing the meaning of a spoken utterance using semantic entity labels. In this paper, we develo…
AVLnet: Learning Audio-Visual Language Representations from Instructional Videos
Andrew Rouditchenko, Angie Boggust, David Harwath +11
Current methods for learning visually grounded language from videos often rely on text annotation, such as human generated captions or machine generated automatic speech recognitio…
Single headed attention based sequence-to-sequence model for state-of-the-art results on Switchboard
Zoltán Tüske, George Saon, Kartik Audhkhasi +1
It is generally believed that direct sequence-to-sequence (seq2seq) speech recognition models are competitive with hybrid models only when a large amount of data, at least a thousa…
Challenging the Boundaries of Speech Recognition: The MALACH Corpus
Michael Picheny, Zóltan Tüske, Brian Kingsbury +3
There has been huge progress in speech recognition over the last several years. Tasks once thought extremely difficult, such as SWITCHBOARD, now approach levels of human performanc…
Guiding CTC Posterior Spike Timings for Improved Posterior Fusion and Knowledge Distillation
Gakuto Kurata, Kartik Audhkhasi
Conventional automatic speech recognition (ASR) systems trained from frame-level alignments can easily leverage posterior fusion to improve ASR accuracy and build a better single m…
Acoustically Grounded Word Embeddings for Improved Acoustics-to-Word Speech Recognition
Shane Settle, Kartik Audhkhasi, Karen Livescu +1
Direct acoustics-to-word (A2W) systems for end-to-end automatic speech recognition are simpler to train, and more efficient to decode with, than sub-word systems. However, A2W syst…
Joint Modeling of Accents and Acoustics for Multi-Accent Speech Recognition
Xuesong Yang, Kartik Audhkhasi, Andrew Rosenberg +3
The performance of automatic speech recognition systems degrades with increasing mismatch between the training and testing scenarios. Differences in speaker accents are a significa…
Building competitive direct acoustics-to-word models for English conversational speech recognition
Kartik Audhkhasi, Brian Kingsbury, Bhuvana Ramabhadran +2
Direct acoustics-to-word (A2W) models in the end-to-end paradigm have received increasing attention compared to conventional sub-word based automatic speech recognition models usin…
Direct Acoustics-to-Word Models for English Conversational Speech Recognition
Kartik Audhkhasi, Bhuvana Ramabhadran, George Saon +2
Recent work on end-to-end automatic speech recognition (ASR) has shown that the connectionist temporal classification (CTC) loss can be used to convert acoustics to phone or charac…
English Conversational Telephone Speech Recognition by Humans and Machines
George Saon, Gakuto Kurata, Tom Sercu +9
One of the most difficult speech recognition tasks is accurate recognition of human to human communication. Advances in deep learning over the last few years have produced major sp…
End-to-End ASR-free Keyword Search from Speech
Kartik Audhkhasi, Andrew Rosenberg, Abhinav Sethy +2
End-to-end (E2E) systems have achieved competitive results compared to conventional hybrid hidden Markov model (HMM)-deep neural network based automatic speech recognition (ASR) sy…
Invariant Representations for Noisy Speech Recognition
Dmitriy Serdyuk, Kartik Audhkhasi, Philémon Brakel +3
Modern automatic speech recognition (ASR) systems need to be robust under acoustic variability arising from environmental, speaker, channel, and recording conditions. Ensuring such…
Diverse Embedding Neural Network Language Models
Kartik Audhkhasi, Abhinav Sethy, Bhuvana Ramabhadran
We propose Diverse Embedding Neural Network (DENN), a novel architecture for language models (LMs). A DENNLM projects the input word history vector onto multiple diverse low-dimens…
Generalized Ambiguity Decomposition for Understanding Ensemble Diversity
Kartik Audhkhasi, Abhinav Sethy, Bhuvana Ramabhadran +1
Diversity or complementarity of experts in ensemble pattern recognition and information processing systems is widely-observed by researchers to be crucial for achieving performance…