7 citations · 11 across the 13 of their papers we have counts for
16 papers
Preference-Aware Pareto Steering for Multi-Task Speech LLMs
Liuyuan Jiang, Vishal Sunder, Samuel Thomas +4
Speech large language models (LLMs) perform multiple tasks, including automatic speech recognition (ASR), timestamp prediction (TS), and speech translation (ST). Joint training of…
Design of the IBM Granite 5.0 TurboCTC ASR Model
Brian Kingsbury, George Saon, Masayuki Suzuki +5
We describe the architecture, training methodology and inference speedups of Granite 5.0 Turbo CTC, a 470 million parameter encoder-only model with an excellent speed-accuracy trad…
Word Timestamps and Speaker Attribution with a Non-Autoregressive LLM
Zvi Kons, Avihu Dekel, Hagai Aronowitz +2
Timestamps and speaker attribution are useful additions to speech recognition, creating a rich text transcript. This information can either be extracted during transcription or ali…
CHILDES-Aligned: A Curated Children's Speech Dataset via Multi-Model Timestamp Ensembling
Haolong Zheng, Yuanzhuo Hu, Xinyu Liang +7
CHILDES is a large-scale child speech corpus containing long-form recordings of naturalistic child-adult interactions, making it a valuable resource for studying child speech and l…
Granite-speech: open-source speech-aware LLMs with strong English ASR capabilities
George Saon, Avihu Dekel, Alexander Brooks +21
Granite-speech LLMs are compact and efficient speech language models specifically designed for English ASR and automatic speech translation (AST). The models were trained by modali…
A Non-autoregressive Model for Joint STT and TTS
Vishal Sunder, Brian Kingsbury, George Saon +5
In this paper, we take a step towards jointly modeling automatic speech recognition (STT) and speech synthesis (TTS) in a fully non-autoregressive way. We develop a novel multimoda…
Improving Transducer-Based Spoken Language Understanding with Self-Conditioned CTC and Knowledge Transfer
Vishal Sunder, Eric Fosler-Lussier
In this paper, we propose to improve end-to-end (E2E) spoken language understand (SLU) in an RNN transducer model (RNN-T) by incorporating a joint self-conditioned CTC automatic sp…
End-to-End real time tracking of children's reading with pointer network
Vishal Sunder, Beulah Karrolla, Eric Fosler-Lussier
In this work, we explore how a real time reading tracker can be built efficiently for children's voices. While previously proposed reading trackers focused on ASR-based cascaded ap…
Towards End-to-End Integration of Dialog History for Improved Spoken Language Understanding
Vishal Sunder, Samuel Thomas, Hong-Kwang J. Kuo +3
Dialog history plays an important role in spoken language understanding (SLU) performance in a dialog system. For end-to-end (E2E) SLU, previous work has used dialog history in tex…
Tokenwise Contrastive Pretraining for Finer Speech-to-BERT Alignment in End-to-End Speech-to-Intent Systems
Vishal Sunder, Eric Fosler-Lussier, Samuel Thomas +2
Recent advances in End-to-End (E2E) Spoken Language Understanding (SLU) have been primarily due to effective pretraining of speech representations. One such pretraining paradigm is…
Building an ASR Error Robust Spoken Virtual Patient System in a Highly Class-Imbalanced Scenario Without Speech Data
Vishal Sunder, Prashant Serai, Eric Fosler-Lussier
A Virtual Patient (VP) is a powerful tool for training medical students to take patient histories, where responding to a diverse set of spoken questions is essential to simulate na…
Hallucination of speech recognition errors with sequence to sequence learning
Prashant Serai, Vishal Sunder, Eric Fosler-Lussier
Automatic Speech Recognition (ASR) is an imperfect process that results in certain mismatches in ASR output text when compared to plain written text or transcriptions. When plain t…
Handling Class Imbalance in Low-Resource Dialogue Systems by Combining Few-Shot Classification and Interpolation
Vishal Sunder, Eric Fosler-Lussier
Utterance classification performance in low-resource dialogue systems is constrained by an inevitably high degree of data imbalance in class labels. We present a new end-to-end pai…
One-shot Information Extraction from Document Images using Neuro-Deductive Program Synthesis
Vishal Sunder, Ashwin Srinivasan, Lovekesh Vig +2
Our interest in this paper is in meeting a rapidly growing industrial demand for information extraction from images of documents such as invoices, bills, receipts etc. In practice…
Prosocial or Selfish? Agents with different behaviors for Contract Negotiation using Reinforcement Learning
Vishal Sunder, Lovekesh Vig, Arnab Chatterjee +1
We present an effective technique for training deep learning agents capable of negotiating on a set of clauses in a contract agreement using a simple communication protocol. We use…
CIKM AnalytiCup 2017 Lazada Product Title Quality Challenge An Ensemble of Deep and Shallow Learning to predict the Quality of Product Titles
Karamjit Singh, Vishal Sunder
We present an approach where two different models (Deep and Shallow) are trained separately on the data and a weighted average of the outputs is taken as the final result. For the…