10 papers
Document Optimization for Black-Box Retrieval via Reinforcement Learning
Omri Uzan, Ron Polonsky, Douwe Kiela +1
Document expansion is a classical technique for improving retrieval quality, and is attractive since it shifts computation offline, avoiding additional query-time processing. Howev…
Steerable Cultural Preference Optimization of Reward Models
Minsik Oh, Advit Deepak, Sophie Wu +2
It is essential for large language model (LLM) technology to serve many different cultural sub-communities in a manner that is acceptable to each community. However, research on LL…
BlitzRank: Principled Zero-shot Ranking Agents with Tournament Graphs
Sheshansh Agrawal, Thien Hang Nguyen, Douwe Kiela
Selecting the top from items via expensive -wise comparisons is central to settings ranging from LLM-based document reranking to crowdsourced evaluation and tournament d…
Single-Agent LLMs Outperform Multi-Agent Systems on Multi-Hop Reasoning Under Equal Thinking Token Budgets
Dat Tran, Douwe Kiela
Recent work reports strong performance from multi-agent LLM systems (MAS), but these gains are often confounded by increased test-time computation. When computation is normalized,…
Reflective Context Learning: Studying the Optimization Primitives of Context Space
Nikita Vassilyev, William Berrios, Ruowang Zhang +3
Generally capable agents must learn from experience in ways that generalize across tasks and environments. The fundamental problems of learning, including credit assignment, overfi…
LMUnit: Fine-grained Evaluation with Natural Language Unit Tests
Jon Saad-Falcon, Rajan Vivek, William Berrios +6
As language models become integral to critical workflows, assessing their behavior remains a fundamental challenge -- human evaluation is costly and noisy, while automated metrics…