BlitzRank: Principled Zero-shot Ranking Agents with Tournament Graphs
arXiv:2602.05448
Abstract
Selecting the top from items via expensive -wise comparisons is central to settings ranging from LLM-based document reranking to crowdsourced evaluation and tournament design. Existing methods either rely on heuristics that discard comparison information, or exploit it at prohibitive cost. We introduce a tournament graph framework that provides a principled foundation for -wise ranking. Our key observation is that each -item comparison reveals an induced tournament of pairwise preferences; aggregating these into a global preference graph and computing its transitive closure yields many additional orderings without further oracle calls. We formalize when the current top- output is certifiably determined and design a greedy query schedule that maximizes information gain towards identifying the top- items. The framework also gracefully handles non-transitive preferences -- cycles induced by real-world oracles -- by collapsing them into equivalence classes that yield principled tiered rankings. Applied to LLM reranking across 14 benchmarks and 5 models, BlitzRank achieves Pareto dominance over existing approaches: matching or exceeding accuracy while requiring 25--40% fewer tokens than comparable methods; against pairwise reranking, it achieves near-identical quality with 7 fewer tokens. Code available at https://github.com/ContextualAI/BlitzRank.
ICML 2026 spotlight