2.3k citations · 2.8k across the 30 of their papers we have counts for
7 papers · 1 filter
Drowning in Documents: Consequences of Scaling Reranker Inference
Mathew Jacob, Erik Lindgren, Matei Zaharia +3
Rerankers, typically cross-encoders, are computationally intensive but are frequently used because they are widely assumed to outperform cheaper initial IR systems. We challenge th…
Fine-Tuning and Prompt Optimization: Two Great Steps that Work Better Together
Dilara Soylu, Christopher Potts, Omar Khattab
Natural Language Processing (NLP) systems are increasingly taking the form of sophisticated modular pipelines, e.g., Retrieval Augmented Generation (RAG), where each module may inv…
Prompts as Auto-Optimized Training Hyperparameters: Training Best-in-Class IR Models from Scratch with 10 Gold Labels
Jasper Xian, Saron Samuel, Faraz Khoubsirat +7
We develop a method for training small-scale (under 100M parameter) neural information retrieval models with as few as 10 gold relevance labels. The method depends on generating sy…
Optimizing Instructions and Demonstrations for Multi-Stage Language Model Programs
Krista Opsahl-Ong, Michael J Ryan, Josh Purtell +4
Language Model Programs, i.e. sophisticated pipelines of modular language model (LM) calls, are increasingly advancing NLP tasks, but they require crafting prompts that are jointly…
When Benchmarks are Targets: Revealing the Sensitivity of Large Language Model Leaderboards
Norah Alzahrani, Hisham Abdullah Alyahya, Yazeed Alnumay +9
Large Language Model (LLM) leaderboards based on benchmark rankings are regularly used to guide practitioners in model selection. Often, the published leaderboard rankings are take…
In-Context Learning for Extreme Multi-Label Classification
Karel D'Oosterlinck, Omar Khattab, François Remy +3
Multi-label classification problems with thousands of classes are hard to solve with in-context learning alone, as language models (LMs) might lack prior knowledge about the precis…
Building Efficient and Effective OpenQA Systems for Low-Resource Languages
Emrah Budur, Rıza Özçelik, Dilara Soylu +3
Question answering (QA) is the task of answering questions posed in natural language with free-form natural language answers extracted from a given passage. In the OpenQA variant,…