23 citations · 37 across the 9 of their papers we have counts for
10 papers
RoPE Distinguishes Neither Positions Nor Tokens in Long Contexts, Provably
Yufeng Du, Phillip Harris, Minyang Tian +5
We identify intrinsic limitations of Rotary Positional Embeddings (RoPE) in Transformer-based long-context language models. Our theoretical analysis abstracts away from the specifi…
Context Length Alone Hurts LLM Performance Despite Perfect Retrieval
Yufeng Du, Minyang Tian, Srikanth Ronanki +7
Large language models (LLMs) often fail to scale their performance on long-context tasks performance in line with the context lengths they support. This gap is commonly attributed…
Low-Resource Compositional Semantic Parsing with Concept Pretraining
Subendhu Rongali, Mukund Sridhar, Haidar Khan +3
Semantic parsing plays a key role in digital voice assistants such as Alexa, Siri, and Google Assistant by mapping natural language to structured meaning representations. When we w…
Training Naturalized Semantic Parsers with Very Little Data
Subendhu Rongali, Konstantine Arkoudas, Melanie Rubino +1
Semantic parsing is an important NLP problem, particularly for voice assistants such as Alexa and Google Assistant. State-of-the-art (SOTA) semantic parsers are seq2seq architectur…
Improved Latent Tree Induction with Distant Supervision via Span Constraints
Zhiyang Xu, Andrew Drozdov, Jay Yoon Lee +6
For over thirty years, researchers have developed and analyzed methods for latent tree induction as an approach for unsupervised syntactic parsing. Nonetheless, modern systems stil…
Exploring Transfer Learning For End-to-End Spoken Language Understanding
Subendhu Rongali, Beiye Liu, Liwei Cai +3
Voice Assistants such as Alexa, Siri, and Google Assistant typically use a two-stage Spoken Language Understanding pipeline; first, an Automatic Speech Recognition (ASR) component…
Compressing Transformer-Based Semantic Parsing Models using Compositional Code Embeddings
Prafull Prakash, Saurabh Kumar Shashidhar, Wenlong Zhao +3
The current state-of-the-art task-oriented semantic parsing models use BERT or RoBERTa as pretrained encoders; these models have huge memory footprints. This poses a challenge to t…
Continual Domain-Tuning for Pretrained Language Models
Subendhu Rongali, Abhyuday Jagannatha, Bhanu Pratap Singh Rawat +1
Pre-trained language models (LM) such as BERT, DistilBERT, and RoBERTa can be tuned for different domains (domain-tuning) by continuing the pre-training phase on a new target domai…
Don't Parse, Generate! A Sequence to Sequence Architecture for Task-Oriented Semantic Parsing
Subendhu Rongali, Luca Soldaini, Emilio Monti +1
Virtual assistants such as Amazon Alexa, Apple Siri, and Google Assistant often rely on a semantic parsing component to understand which action(s) to execute for an utterance spoke…
Taxonomy grounded aggregation of classifiers with different label sets
Amrita Saha, Sathish Indurthi, Shantanu Godbole +2
We describe the problem of aggregating the label predictions of diverse classifiers using a class taxonomy. Such a taxonomy may not have been available or referenced when the indiv…