3 papers
cs.LG2025
Retrieval Capabilities of Large Language Models Scale with Pretraining FLOPs
Jacob Portes, Connor Jennings, Erica Ji Yuen +2
How does retrieval performance scale with pretraining FLOPs? We benchmark retrieval performance across LLM model sizes from 125 million parameters to 7 billion parameters pretraine…
cs.IR2025
Drowning in Documents: Consequences of Scaling Reranker Inference
Mathew Jacob, Erik Lindgren, Matei Zaharia +3
Rerankers, typically cross-encoders, are computationally intensive but are frequently used because they are widely assumed to outperform cheaper initial IR systems. We challenge th…
cs.IR2025
FreshStack: Building Realistic Benchmarks for Evaluating Retrieval on Technical Documents
Nandan Thakur, Jimmy Lin, Sam Havens +3
We introduce FreshStack, a holistic framework for automatically building information retrieval (IR) evaluation benchmarks by incorporating challenging questions and answers. FreshS…