6 papers
: Semantic Residual Effective Contrastive Information for Evaluating Hard Negatives
Aarush Sinha, Rahul Seetharaman, Aman Bansal
Hard-negative source selection for dense retrieval is usually decided only after fine-tuning and downstream evaluation. We propose ECIsem, a validity-weighted diagnostic that ranks…
Scaling Laws for Cross-Encoder Reranking
Rahul Seetharaman, Aman Bansal, Hamed Zamani +1
Scaling laws are well studied for language models and first-stage retrieval, but not for reranking. We present the first systematic study of scaling laws for cross-encoder reranker…
Leveraging Large Language Models for Predictive Analysis of Human Misery
Bishanka Seal, Rahul Seetharaman, Aman Bansal +1
This study investigates the use of Large Language Models (LLMs) for predicting human-perceived misery scores from natural language descriptions of real-world scenarios. The task is…
InsertRank: LLMs can reason over BM25 scores to Improve Listwise Reranking
Rahul Seetharaman, Kaustubh D. Dhole, Aman Bansal
Large Language Models (LLMs) have demonstrated significant strides across various information retrieval tasks, particularly as rerankers, owing to their strong generalization and k…
ViBe: A Text-to-Video Benchmark for Evaluating Hallucination in Large Multimodal Models
Vipula Rawte, Sarthak Jain, Aarush Sinha +9
Recent advances in Large Multimodal Models (LMMs) have expanded their capabilities to video understanding, with Text-to-Video (T2V) models excelling in generating videos from textu…
YesBut: A High-Quality Annotated Multimodal Dataset for evaluating Satire Comprehension capability of Vision-Language Models
Abhilash Nandy, Yash Agarwal, Ashish Patwa +5
Understanding satire and humor is a challenging task for even current Vision-Language models. In this paper, we propose the challenging tasks of Satirical Image Detection (detectin…