9 papers
Can Language Models Actually Retrieve In-Context? Drowning in Documents at Million Token Scale
Siddharth Gollapudi, Nilesh Gupta, Prasann Singhal +1
Language models (LMs) raise an intriguing alternative to vector-based retrieval: conditioning on an in-context corpus and directly generating a relevant answer. However, prior work…
LLM-guided Hierarchical Search for End-to-end Reasoning Intensive Retrieval
Nilesh Gupta, Wei-Cheng Chang, Ngot Bui +2
Search systems are increasingly used for reasoning-intensive queries, where what makes a document relevant requires understanding or reasoning over the query-document relation rath…
Compute Allocation for Reasoning-Intensive Retrieval Agents
Sreeja Apparaju, Nilesh Gupta
As agents operate over long horizons, their memory stores grow continuously, making retrieval critical to accessing relevant information. Many agent queries require reasoning-inten…
LUCID: Attention with Preconditioned Representations
Sai Surya Duvvuri, Nirmal Patel, Nilesh Gupta +1
Softmax-based dot-product attention is a cornerstone of Transformer architectures, enabling remarkable capabilities such as in-context learning. However, as context lengths increas…
Autoregressive Ranking: Bridging the Gap Between Dual and Cross Encoders
Benjamin Rozonoyer, Chong You, Michael Boratko +5
The success of Large Language Models (LLMs) has motivated a shift toward generative approaches to retrieval and ranking, aiming to supersede classical Dual Encoders (DEs) and Cross…
Compressing Many-Shots in In-Context Learning
Devvrit Khatri, Pranamya Kulkarni, Nilesh Gupta +9
Large Language Models (LLMs) have been shown to be able to learn different tasks without explicit finetuning when given many input-output examples / demonstrations through In-Conte…