38 citations · 80 across the 14 of their papers we have counts for
14 papers
Sharing is Caring: Efficient LM Post-Training with Collective RL Experience Sharing
Jeffrey Amico, Gabriel Passamani Andrade, John Donaghy +12
Post-training language models (LMs) with reinforcement learning (RL) can enhance their complex reasoning capabilities without supervised fine-tuning, as demonstrated by DeepSeek-R1…
NoLoCo: No-all-reduce Low Communication Training Method for Large Models
Jari Kolehmainen, Nikolay Blagoev, Semih Kara +3
Training large language models is generally done on clusters containing thousands of accelerators, communicating over a high-bandwidth interconnect. Scaling up these clusters is ex…
HDEE: Heterogeneous Domain Expert Ensemble
Oğuzhan Ersoy, Jari Kolehmainen, Gabriel Passamani Andrade
Training dense LLMs requires enormous amounts of data and centralized compute, which introduces fundamental bottlenecks and ever-growing costs for large models. Several studies aim…
Speech Recognition Rescoring with Large Speech-Text Foundation Models
Prashanth Gurunath Shivakumar, Jari Kolehmainen, Aditya Gourav +4
Large language models (LLM) have demonstrated the ability to understand human language by leveraging large amount of text data. Automatic speech recognition (ASR) systems are often…
Multi-Modal Retrieval For Large Language Model Based Speech Recognition
Jari Kolehmainen, Aditya Gourav, Prashanth Gurunath Shivakumar +5
Retrieval is a widely adopted approach for improving language models leveraging external information. As the field moves towards multi-modal large language models, it is important…
Investigating Training Strategies and Model Robustness of Low-Rank Adaptation for Language Modeling in Speech Recognition
Yu Yu, Chao-Han Huck Yang, Tuan Dinh +10
The use of low-rank adaptation (LoRA) with frozen pretrained language models (PLMs) has become increasing popular as a mainstream, resource-efficient modeling approach for memory-c…
Towards ASR Robust Spoken Language Understanding Through In-Context Learning With Word Confusion Networks
Kevin Everson, Yile Gu, Huck Yang +10
In the realm of spoken language understanding (SLU), numerous natural language understanding (NLU) methodologies have been adapted by supplying large language models (LLMs) with tr…
Discriminative Speech Recognition Rescoring with Pre-trained Language Models
Prashanth Gurunath Shivakumar, Jari Kolehmainen, Yile Gu +3
Second pass rescoring is a critical component of competitive automatic speech recognition (ASR) systems. Large language models have demonstrated their ability in using pre-trained…
Low-rank Adaptation of Large Language Model Rescoring for Parameter-Efficient Speech Recognition
Yu Yu, Chao-Han Huck Yang, Jari Kolehmainen +15
We propose a neural language modeling system based on low-rank adaptation (LoRA) for speech recognition output rescoring. Although pretrained language models (LMs) like BERT have s…
Personalization for BERT-based Discriminative Speech Recognition Rescoring
Jari Kolehmainen, Yile Gu, Aditya Gourav +4
Recognition of personalized content remains a challenge in end-to-end speech recognition. We explore three novel approaches that use personalized content in a neural rescoring step…
Scaling Laws for Discriminative Speech Recognition Rescoring Models
Yile Gu, Prashanth Gurunath Shivakumar, Jari Kolehmainen +3
Recent studies have found that model performance has a smooth power-law relationship, or scaling laws, with training data and model size, for a wide range of problems. These scalin…
Distillation Strategies for Discriminative Speech Recognition Rescoring
Prashanth Gurunath Shivakumar, Jari Kolehmainen, Yile Gu +3
Second-pass rescoring is employed in most state-of-the-art speech recognition systems. Recently, BERT based models have gained popularity for re-ranking the n-best hypothesis by ex…
RescoreBERT: Discriminative Speech Recognition Rescoring with BERT
Liyan Xu, Yile Gu, Jari Kolehmainen +5
Second-pass rescoring is an important component in automatic speech recognition (ASR) systems that is used to improve the outputs from a first-pass decoder by implementing a lattic…
Charge Transport Equation for Bidisperse Collisional Granular Flows with Nonequipartitioned Fluctuating Kinetic Energy
Lise Ceresiat, Jari Kolehmainen, Ali Ozel
Starting from the Boltzmann-Enskog kinetic equations, the charge transport equation for bidisperse granular flows with contact electrification is derived with separate mean velocit…