38 citations · 124 across the 30 of their papers we have counts for
32 papers
Improving Text-to-Audio Instruction Following via Fine-Grained Feedback from Audio-Aware Large Language Models
Chun-Yi Kuan, Siwon Kim, Byeonggeun Kim +7
Recent text-to-audio models generate high-quality audio, but often fail to follow instructions involving multiple sound events and temporal order. This gap arises because existing…
Incentivizing Consistent, Effective and Scalable Reasoning Capability in Audio LLMs via Reasoning Process Rewards
Jiajun Fan, Roger Ren, Jingyuan Li +6
The role of reasoning in Audio Large Language Models remains widely underexplored, as introducing a reasoning process often degrades rather than improves performance during inferen…
Group Relative Policy Optimization for Speech Recognition
Prashanth Gurunath Shivakumar, Yile Gu, Ankur Gandhe +1
Speech Recognition has seen a dramatic shift towards adopting Large Language Models (LLMs). This shift is partly driven by good scalability properties demonstrated by LLMs, ability…
Align-SLM: Textless Spoken Language Models with Reinforcement Learning from AI Feedback
Guan-Ting Lin, Prashanth Gurunath Shivakumar, Aditya Gourav +4
While textless Spoken Language Models (SLMs) have shown potential in end-to-end speech-to-speech modeling, they still lag behind text-based Large Language Models (LLMs) in terms of…
Speech Recognition Rescoring with Large Speech-Text Foundation Models
Prashanth Gurunath Shivakumar, Jari Kolehmainen, Aditya Gourav +4
Large language models (LLM) have demonstrated the ability to understand human language by leveraging large amount of text data. Automatic speech recognition (ASR) systems are often…
Multi-Modal Retrieval For Large Language Model Based Speech Recognition
Jari Kolehmainen, Aditya Gourav, Prashanth Gurunath Shivakumar +5
Retrieval is a widely adopted approach for improving language models leveraging external information. As the field moves towards multi-modal large language models, it is important…
Investigating Training Strategies and Model Robustness of Low-Rank Adaptation for Language Modeling in Speech Recognition
Yu Yu, Chao-Han Huck Yang, Tuan Dinh +10
The use of low-rank adaptation (LoRA) with frozen pretrained language models (PLMs) has become increasing popular as a mainstream, resource-efficient modeling approach for memory-c…
Paralinguistics-Enhanced Large Language Modeling of Spoken Dialogue
Guan-Ting Lin, Prashanth Gurunath Shivakumar, Ankur Gandhe +6
Large Language Models (LLMs) have demonstrated superior abilities in tasks such as chatting, reasoning, and question-answering. However, standard LLMs may ignore crucial paralingui…
Towards ASR Robust Spoken Language Understanding Through In-Context Learning With Word Confusion Networks
Kevin Everson, Yile Gu, Huck Yang +10
In the realm of spoken language understanding (SLU), numerous natural language understanding (NLU) methodologies have been adapted by supplying large language models (LLMs) with tr…
Discriminative Speech Recognition Rescoring with Pre-trained Language Models
Prashanth Gurunath Shivakumar, Jari Kolehmainen, Yile Gu +3
Second pass rescoring is a critical component of competitive automatic speech recognition (ASR) systems. Large language models have demonstrated their ability in using pre-trained…
Low-rank Adaptation of Large Language Model Rescoring for Parameter-Efficient Speech Recognition
Yu Yu, Chao-Han Huck Yang, Jari Kolehmainen +15
We propose a neural language modeling system based on low-rank adaptation (LoRA) for speech recognition output rescoring. Although pretrained language models (LMs) like BERT have s…
Personalization for BERT-based Discriminative Speech Recognition Rescoring
Jari Kolehmainen, Yile Gu, Aditya Gourav +4
Recognition of personalized content remains a challenge in end-to-end speech recognition. We explore three novel approaches that use personalized content in a neural rescoring step…
Scaling Laws for Discriminative Speech Recognition Rescoring Models
Yile Gu, Prashanth Gurunath Shivakumar, Jari Kolehmainen +3
Recent studies have found that model performance has a smooth power-law relationship, or scaling laws, with training data and model size, for a wide range of problems. These scalin…
Distillation Strategies for Discriminative Speech Recognition Rescoring
Prashanth Gurunath Shivakumar, Jari Kolehmainen, Yile Gu +3
Second-pass rescoring is employed in most state-of-the-art speech recognition systems. Recently, BERT based models have gained popularity for re-ranking the n-best hypothesis by ex…
Streaming Speech-to-Confusion Network Speech Recognition
Denis Filimonov, Prabhat Pandey, Ariya Rastrow +2
In interactive automatic speech recognition (ASR) systems, low-latency requirements limit the amount of search space that can be explored during decoding, particularly in end-to-en…
Robust Acoustic and Semantic Contextual Biasing in Neural Transducers for Speech Recognition
Xuandi Fu, Kanthashree Mysore Sathyendra, Ankur Gandhe +4
Attention-based contextual biasing approaches have shown significant improvements in the recognition of generic and/or personal rare-words in End-to-End Automatic Speech Recognitio…
PROCTER: PROnunciation-aware ConTextual adaptER for personalized speech recognition in neural transducers
Rahul Pandey, Roger Ren, Qi Luo +7
End-to-End (E2E) automatic speech recognition (ASR) systems used in voice assistants often have difficulties recognizing infrequent words personalized to the user, such as names an…
On-the-fly Text Retrieval for End-to-End ASR Adaptation
Bolaji Yusuf, Aditya Gourav, Ankur Gandhe +1
End-to-end speech recognition models are improved by incorporating external text sources, typically by fusion with an external language model. Such language models have to be retra…
RescoreBERT: Discriminative Speech Recognition Rescoring with BERT
Liyan Xu, Yile Gu, Jari Kolehmainen +5
Second-pass rescoring is an important component in automatic speech recognition (ASR) systems that is used to improve the outputs from a first-pass decoder by implementing a lattic…
USTED: Improving ASR with a Unified Speech and Text Encoder-Decoder
Bolaji Yusuf, Ankur Gandhe, Alex Sokolov
Improving end-to-end speech recognition by incorporating external text data has been a longstanding research topic. There has been a recent focus on training E2E ASR models that ge…
A Likelihood Ratio based Domain Adaptation Method for E2E Models
Chhavi Choudhury, Ankur Gandhe, Xiaohan Ding +1
End-to-end (E2E) automatic speech recognition models like Recurrent Neural Networks Transducer (RNN-T) are becoming a popular choice for streaming ASR applications like voice assis…
Prompt Tuning GPT-2 language model for parameter-efficient domain adaptation of ASR systems
Saket Dingliwal, Ashish Shenoy, Sravan Bodapati +3
Automatic Speech Recognition (ASR) systems have found their use in numerous industrial applications in very diverse domains creating a need to adapt to new domains with small memor…
Lattention: Lattice-attention in ASR rescoring
Prabhat Pandey, Sergio Duarte Torres, Ali Orkan Bayer +2
Lattices form a compact representation of multiple hypotheses generated from an automatic speech recognition system and have been shown to improve performance of downstream tasks l…
Prompt-tuning in ASR systems for efficient domain-adaptation
Saket Dingliwal, Ashish Shenoy, Sravan Bodapati +3
Automatic Speech Recognition (ASR) systems have found their use in numerous industrial applications in very diverse domains. Since domain-specific systems perform better than their…