5 citations · 6 across the 10 of their papers we have counts for
10 papers
CASA: Classification Augmented with Safety Attention for Robust Multimodal Alignment
Anurag Kumar, Raghuveer Peri, Jon Burnsky +4
Multimodal large-language models (MLLMs) often experience degraded safety alignment when harmful queries exploit cross-modal interactions. Models aligned on text alone show a highe…
SEAL: Speaker Error Correction using Acoustic-conditioned Large Language Models
Anurag Kumar, Rohit Paturi, Amber Afshan +1
Speaker Diarization (SD) is a crucial component of modern end-to-end ASR pipelines. Traditional SD systems, which are typically audio-based and operate independently of ASR, often…
Speakers Unembedded: Embedding-free Approach to Long-form Neural Diarization
Xiang Li, Vivek Govindan, Rohit Paturi +1
End-to-end neural diarization (EEND) models offer significant improvements over traditional embedding-based Speaker Diarization (SD) approaches but falls short on generalizing to l…
AG-LSEC: Audio Grounded Lexical Speaker Error Correction
Rohit Paturi, Xiang Li, Sundararajan Srinivasan
Speaker Diarization (SD) systems are typically audio-based and operate independently of the ASR system in traditional speech transcription pipelines and can have speaker errors due…
SpeechVerse: A Large-scale Generalizable Audio Language Model
Nilaksh Das, Saket Dingliwal, Srikanth Ronanki +14
Large language models (LLMs) have shown incredible proficiency in performing tasks that require semantic understanding of natural language instructions. Recently, many works have f…
Generalized zero-shot audio-to-intent classification
Veera Raghavendra Elluru, Devang Kulshreshtha, Rohit Paturi +2
Spoken language understanding systems using audio-only data are gaining popularity, yet their ability to handle unseen intents remains limited. In this study, we propose a generali…
End-to-End Single-Channel Speaker-Turn Aware Conversational Speech Translation
Juan Zuluaga-Gomez, Zhaocheng Huang, Xing Niu +5
Conventional speech-to-text translation (ST) systems are trained on single-speaker utterances, and they may not generalize to real-life scenarios where the audio contains conversat…
Speaker Diarization of Scripted Audiovisual Content
Yogesh Virkar, Brian Thompson, Rohit Paturi +2
The media localization industry usually requires a verbatim script of the final film or TV production in order to create subtitles or dubbing scripts in a foreign language. In part…
Lexical Speaker Error Correction: Leveraging Language Models for Speaker Diarization Error Correction
Rohit Paturi, Sundararajan Srinivasan, Xiang Li
Speaker diarization (SD) is typically used with an automatic speech recognition (ASR) system to ascribe speaker labels to recognized words. The conventional approach reconciles out…
Directed Speech Separation for Automatic Speech Recognition of Long Form Conversational Speech
Rohit Paturi, Sundararajan Srinivasan, Katrin Kirchhoff +1
Many of the recent advances in speech separation are primarily aimed at synthetic mixtures of short audio utterances with high degrees of overlap. Most of these approaches need an…