27 citations · 69 across the 35 of their papers we have counts for
24 papers · 1 filter
DualSQL: Text-to-SQL with Multi-Agent Reinforcement Learning
Shijie Chen, Yu Gan, Yeounoh Chung +6
State-of-the-art Text-to-SQL systems are typically multi-agent pipelines centered around two fundamental tasks: schema linking and SQL generation. However, existing work trains sep…
SGD-KV: Summarization Guided KV Cache Compression
Zeyu Liu, Woomin Song, Xuandi Fu +5
Large language models (LLMs) face severe memory bottlenecks in long-context inference due to the linearly growing size of key-value (KV) caches. Existing KV cache compression techn…
Context Length Alone Hurts LLM Performance Despite Perfect Retrieval
Yufeng Du, Minyang Tian, Srikanth Ronanki +7
Large language models (LLMs) often fail to scale their performance on long-context tasks performance in line with the context lengths they support. This gap is commonly attributed…
LAWCAT: Efficient Distillation from Quadratic to Linear Attention with Convolution across Tokens for Long Context Modeling
Zeyu Liu, Souvik Kundu, Lianghao Jiang +5
Although transformer architectures have achieved state-of-the-art performance across diverse domains, their quadratic computational complexity with respect to sequence length remai…
Accelerated Test-Time Scaling with Model-Free Speculative Sampling
Woomin Song, Saket Dingliwal, Sai Muralidhar Jayanthi +4
Language models have demonstrated remarkable capabilities in reasoning tasks through test-time scaling techniques like best-of-N sampling and tree search. However, these approaches…
Compress, Gather, and Recompute: REFORMing Long-Context Processing in Transformers
Woomin Song, Sai Muralidhar Jayanthi, Srikanth Ronanki +5
As large language models increasingly gain popularity in real-world applications, processing extremely long contexts, often exceeding the model's pre-trained context limits, has em…
Mamba Drafters for Speculative Decoding
Daewon Choi, Seunghyuk Oh, Saket Dingliwal +9
Speculative decoding has emerged as a promising approach to accelerating large language model (LLM) generation using a fast drafter while maintaining alignment with the target mode…
ConSiDERS-The-Human Evaluation Framework: Rethinking Human Evaluation for Generative Large Language Models
Aparna Elangovan, Ling Liu, Lei Xu +2
In this position paper, we argue that human evaluation of generative large language models (LLMs) should be a multidisciplinary undertaking that draws upon insights from discipline…
Multi-teacher Distillation for Multilingual Spelling Correction
Jingfen Zhang, Xuan Guo, Sravan Bodapati +1
Accurate spelling correction is a critical step in modern search interfaces, especially in an era of mobile devices and speech-to-text interfaces. For services that are deployed ar…
Retrieve and Copy: Scaling ASR Personalization to Large Catalogs
Sai Muralidhar Jayanthi, Devang Kulshreshtha, Saket Dingliwal +2
Personalization of automatic speech recognition (ASR) models is a widely studied topic because of its many practical applications. Most recently, attention-based contextual biasing…
Multilingual Contextual Adapters To Improve Custom Word Recognition In Low-resource Languages
Devang Kulshreshtha, Saket Dingliwal, Brady Houston +1
Connectionist Temporal Classification (CTC) models are popular for their balance between speed and performance for Automatic Speech Recognition (ASR). However, these CTC models sti…
Don't Stop Self-Supervision: Accent Adaptation of Speech Representations via Residual Adapters
Anshu Bhatia, Sanchit Sinha, Saket Dingliwal +3
Speech representations learned in a self-supervised fashion from massive unlabeled speech corpora have been adapted successfully toward several downstream tasks. However, such repr…
Rethinking the Role of Scale for In-Context Learning: An Interpretability-based Case Study at 66 Billion Scale
Hritik Bansal, Karthik Gopalakrishnan, Saket Dingliwal +3
Language models have been shown to perform better with an increase in scale on a wide variety of tasks via the in-context learning paradigm. In this paper, we investigate the hypot…
Device Directedness with Contextual Cues for Spoken Dialog Systems
Dhanush Bekal, Sundararajan Srinivasan, Sravan Bodapati +2
In this work, we define barge-in verification as a supervised learning task where audio-only information is used to classify user spoken dialogue into true and false barge-ins. Fol…
Towards Personalization of CTC Speech Recognition Models with Contextual Adapters and Adaptive Boosting
Saket Dingliwal, Monica Sunkara, Sravan Bodapati +3
End-to-end speech recognition models trained using joint Connectionist Temporal Classification (CTC)-Attention loss have gained popularity recently. In these models, a non-autoregr…
Prompt Tuning GPT-2 language model for parameter-efficient domain adaptation of ASR systems
Saket Dingliwal, Ashish Shenoy, Sravan Bodapati +3
Automatic Speech Recognition (ASR) systems have found their use in numerous industrial applications in very diverse domains creating a need to adapt to new domains with small memor…
Prompt-tuning in ASR systems for efficient domain-adaptation
Saket Dingliwal, Ashish Shenoy, Sravan Bodapati +3
Automatic Speech Recognition (ASR) systems have found their use in numerous industrial applications in very diverse domains. Since domain-specific systems perform better than their…
Adapting Long Context NLM for ASR Rescoring in Conversational Agents
Ashish Shenoy, Sravan Bodapati, Monica Sunkara +2
Neural Language Models (NLM), when trained and evaluated with context spanning multiple utterances, have been shown to consistently outperform both conventional n-gram language mod…
Contextual Biasing of Language Models for Speech Recognition in Goal-Oriented Conversational Agents
Ashish Shenoy, Sravan Bodapati, Katrin Kirchhoff
Goal-oriented conversational interfaces are designed to accomplish specific tasks and typically have interactions that tend to span multiple turns adhering to a pre-defined structu…
Neural Inverse Text Normalization
Monica Sunkara, Chaitanya Shivade, Sravan Bodapati +1
While there have been several contributions exploring state of the art techniques for text normalization, the problem of inverse text normalization (ITN) remains relatively unexplo…
Towards Semi-Supervised Semantics Understanding from Speech
Cheng-I Lai, Jin Cao, Sravan Bodapati +1
Much recent work on Spoken Language Understanding (SLU) falls short in at least one of three ways: models were trained on oracle text input and neglected the Automatics Speech Reco…
Robust Prediction of Punctuation and Truecasing for Medical ASR
Monica Sunkara, Srikanth Ronanki, Kalpit Dixit +2
Automatic speech recognition (ASR) systems in the medical domain that focus on transcribing clinical dictations and doctor-patient conversations often pose many challenges due to t…
Robustness to Capitalization Errors in Named Entity Recognition
Sravan Bodapati, Hyokun Yun, Yaser Al-Onaizan
Robustness to capitalization errors is a highly desirable characteristic of named entity recognizers, yet we find standard models for the task are surprisingly brittle to such nois…
Neural Word Decomposition Models for Abusive Language Detection
Sravan Babu Bodapati, Spandana Gella, Kasturi Bhattacharjee +1
User generated text on social media often suffers from a lot of undesired characteristics including hatespeech, abusive language, insults etc. that are targeted to attack or abuse…