9 citations · 34 across the 71 of their papers we have counts for
77 papers
Attention Sinks in VLMs Are Borrowed, Not Seen
Dengjia Zhang, Alexander Martin, Pengyu Yan +6
We study how to detect and mitigate visual attention sinks in vision-language models (VLMs). Visual attention sinks are image tokens that absorb a large share of attention over the…
SIGMA: Self-Improving Alignment Generalization from a Model Spec
Jingyu Zhang, Shruti Palaskar, Daniel Khashabi +3
LLM agents are increasingly capable of executing complex tasks and of recursively improving themselves on easy-to-verify objectives such as software engineering and mathematics. Si…
Information Abundance Paradox: Long-Context Training Undermines Parametric Knowledge
Arda Uzunoglu, Benjamin Van Durme, Daniel Khashabi
Large language models are increasingly trained and deployed with long contexts that span documents, code repositories, and interaction histories. This scaling reflects the implicit…
Rethinking LoRA Memory Through the Lens of KV Cache Compression
Chunsheng Zuo, Liaoyaqi Wang, William Jurayj +2
Parametric retrieval augmentation encodes document information into lightweight, document-specific modules such as LoRA adapters, reducing the need to include all evidence as input…
DAR: Deontic Reasoning with Agentic Harnesses
Guangyao Dou, William Jurayj, Nils Holzenberger +1
Deontic reasoning is the task of answering questions by applying explicit rules and policies to case-specific facts, for example computing tax liability under a statute or determin…
Configurable Reward Model for Balanced Safety Alignment
Zhengping Jiang, Mehran Khodabandeh, Akash Bharadwaj +5
Aligning large language models (LLMs) to heterogeneous and rapidly evolving safety requirements remains a critical challenge. Existing instruction-tuned LLMs and standalone safety…
MARQUIS: A Three-Stage Pipeline for Video Retrieval-Augmented Generation
Debashish Chakraborty, Dengjia Zhang, Jialiang Jin +7
Retrieval-augmented generation from videos requires systems to retrieve relevant audiovisual evidence from large corpora and synthesize it into coherent, attributed text. Current a…
Weird Generalization is Weirdly Brittle
Miriam Wanner, Hannah Collison, William Jurayj +3
Weird generalization is a phenomenon in which models fine-tuned on data from a narrow domain (e.g. insecure code) develop surprising traits that manifest even outside that domain (…
Can Coding Agents Reproduce Findings in Computational Materials Science?
Ziyang Huang, Yi Cao, Ali K. Shargh +15
Large language models are increasingly deployed as autonomous coding agents and have achieved remarkably strong performance on software engineering benchmarks. However, it is uncle…
A Replicability Study of XTR
Rohan Jha, Reno Kriz, Benjamin Van Durme
The XTR (conteXtual Token Retrieval) algorithm is a modification to ColBERT retrieval that avoids the costly step of fully gathering and reranking the candidates' embeddings by imp…
Unified Multimodal Uncertain Inference
Dengjia Zhang, Alexander Martin, William Jurayj +3
We introduce Unified Multimodal Uncertain Inference (UMUI), a multimodal inference task spanning text, audio, and video, where models must produce calibrated probability estimates…
DeonticBench: A Benchmark for Reasoning over Rules
Guangyao Dou, Luis Brena, Akhil Deo +4
Reasoning with complex, context-specific rules remains challenging for large language models (LLMs). In legal and policy settings, this manifests as deontic reasoning: reasoning ab…
Are Finer Citations Always Better? Rethinking Granularity for Attributed Generation
Hexuan Wang, Jingyu Zhang, Benjamin Van Durme +1
Citation granularity -- whether to cite individual sentences, paragraphs, or documents -- is a critical design choice in attributed generation. While fine-grained citations are com…
Process Supervision of Confidence Margin for Calibrated LLM Reasoning
Liaoyaqi Wang, Chunsheng Zuo, William Jurayj +2
Scaling test-time computation with reinforcement learning (RL) has emerged as a reliable path to improve large language models (LLM) reasoning ability. Yet, outcome-based reward of…
Many-Tier Instruction Hierarchy in LLM Agents
Jingyu Zhang, Tianjian Li, William Jurayj +3
Large language model agents receive instructions from many sources-system messages, user prompts, tool outputs, other agents, and more-each carrying different levels of trust and a…
A Brief Comparison of Training-Free Multi-Vector Sequence Compression Methods
Rohan Jha, Chunsheng Zuo, Reno Kriz +1
While multi-vector retrieval models outperform single-vector models of comparable size in retrieval quality, their practicality is limited by substantially larger index sizes, driv…
Does Reasoning Make Search More Fair? Comparing Fairness in Reasoning and Non-Reasoning Rerankers
Saron Samuel, Benjamin Van Durme, Eugene Yang
While reasoning rerankers, such as Rank1, have demonstrated strong abilities in improving ranking relevance, it is unclear how they perform on other retrieval qualities such as fai…
Multi-Vector Index Compression in Any Modality
Hanxiang Qin, Alexander Martin, Rohan Jha +3
We study efficient multi-vector retrieval for late interaction in any modality. Late interaction has emerged as a dominant paradigm for information retrieval in text, images, visua…
RANKVIDEO: Reasoning Reranking for Text-to-Video Retrieval
Tyler Skow, Alexander Martin, Benjamin Van Durme +2
Reranking is a critical component of modern retrieval systems, which typically pair an efficient first-stage retriever with a more expressive model to refine results. While large r…
Conformal Thinking: Risk Control for Reasoning on a Compute Budget
Xi Wang, Anushri Suresh, Alvin Zhang +6
Reasoning Large Language Models (LLMs) enable test-time scaling, with dataset-level accuracy improving as the token budget increases, motivating adaptive reasoning -- spending toke…
HLTCOE Evaluation Team at TREC 2025: VQA Track
Dengjia Zhang, Charles Weng, Katherine Guerrerio +5
The HLTCOE Evaluation team participated in TREC VQA's Answer Generation (AG) task, for which we developed a listwise learning framework that aims to improve semantic precision and…
All Claims Are Equal, but Some Claims Are More Equal Than Others: Importance-Sensitive Factuality Evaluation of LLM Generations
Miriam Wanner, Leif Azzopardi, Paul Thomas +3
Existing methods for evaluating the factuality of large language model (LLM) responses treat all claims as equally important. This results in misleading evaluations when vital info…
SocialNLI: A Dialogue-Centric Social Inference Dataset
Akhil Deo, Kate Sanders, Benjamin Van Durme
Making theory-of-mind inferences from human dialogue is a strong indicator of a model's underlying social abilities, which are fundamental for adept AI assistants. However, large l…
Seeing Through the MiRAGE: Evaluating Multimodal Retrieval Augmented Generation
Alexander Martin, William Walden, Reno Kriz +5
We introduce MiRAGE, an evaluation framework for retrieval-augmented generation (RAG) from multimodal sources. As audiovisual media becomes a more prevalent source of information o…