17 papers
Tevatron-Elastic: A Unified Abstraction for Training Elastic Retrievers and Rerankers
Yu Wang, Shengyao Zhuang, Xueguang Ma +4
A single model scale challenges the flexibility of a production retrieval system: some settings need it faster, others need a smaller index, and the right trade-off changes with th…
Tevatron Meets Megatron: Expert-Parallel LLM Reranker Training on an Academic Budget
Zhichao Xu, Xueguang Ma, Shengyao Zhuang +5
Modern reranking recipes---billion-scale cross-encoders, mixture-of-experts (MoE) backbones, and distillation against strong teachers---have outpaced the training infrastructure av…
PERSONAJUDGE: Simulating Individual Human Preference Judgments with Evaluator-Specific Demonstration Data
Zeyu He, Xuan Qi, Subramanian Chidambaram +4
Large language models increasingly serve as judges in AI evaluation, but current approaches rely on consensus preferences that ignore individual evaluator variation. We propose a n…
Rethinking On-policy Optimization for Query Augmentation
Zhichao Xu, Shengyao Zhuang, Xueguang Ma +6
Recent advances in large language models (LLMs) have led to a surge of interest in query augmentation for information retrieval (IR). Two main approaches have emerged. The first pr…
Improving Long-Context Retrieval with Multi-Prefix Embedding
Zhenglin Yu, Xueguang Ma, Shengyao Zhuang +4
Long-context retrieval exposes a tension: single-vector embeddings lose fine-grained detail, while token-level multi-vector methods incur prohibitive storage. We propose Multi-Pref…
An Empirical Study of Automating Agent Evaluation
Kang Zhou, Sangmin Woo, Haibo Ding +14
Agent evaluation requires assessing complex multi-step behaviors involving tool use and intermediate reasoning, making it costly and expertise-intensive. A natural question arises:…