4 papers
Masking or Mitigating? Deconstructing the Impact of Query Rewriting on Retriever Biases in RAG
Agam Goyal, Koyel Mukherjee, Apoorv Saxena +3
Dense retrievers in retrieval-augmented generation (RAG) systems exhibit systematic biases -- including brevity, position, literal matching, and repetition biases -- that can compr…
FlowCast: Trajectory Forecasting for Scalable Zero-Cost Speculative Flow Matching
Divya Jyoti Bajpai, Shubham Agarwal, Apoorv Saxena +3
Flow Matching (FM) has recently emerged as a powerful approach for high-quality visual generation. However, their prohibitively slow inference due to a large number of denoising st…
ContextFocus: Activation Steering for Contextual Faithfulness in Large Language Models
Nikhil Anand, Shwetha Somasundaram, Anirudh Phukan +2
Large Language Models (LLMs) encode vast amounts of parametric knowledge during pre-training. As world knowledge evolves, effective deployment increasingly depends on their ability…
RCStat: A Statistical Framework for using Relative Contextualization in Transformers
Debabrata Mahapatra, Shubham Agarwal, Apoorv Saxena +1
Prior work on input-token importance in auto-regressive transformers has relied on Softmax-normalized attention weights, which obscure the richer structure of pre-Softmax query-key…