6 citations · 17 across the 24 of their papers we have counts for
28 papers
Attention Sinks in VLMs Are Borrowed, Not Seen
Dengjia Zhang, Alexander Martin, Pengyu Yan +6
We study how to detect and mitigate visual attention sinks in vision-language models (VLMs). Visual attention sinks are image tokens that absorb a large share of attention over the…
LAYERSCOPE: A Layerwise Characterization of Video and Multimodal Learned Representations
Sandra Arcos-Holzinger, Debashish Chakraborty, Rohita Mocharla +7
We propose LAYERSCOPE, a label-free, layerwise framework that aims to characterize a model's learned representations in video and multimodal settings. Evaluating downstream perform…
MultiVENT-Raw: A Benchmark for Retrieval and Reasoning over Raw Videos
Reno Kriz, David Etter, Alexander Martin +11
Online information is increasingly consumed in video format. Much of this comes in the form of *raw video*: continuous footage taken on a cell phone, with a hand-held camera, or vi…
Information Satisfaction: A Reader-Centered Axis for Summarization Evaluation
Isabel Cachola, William Walden, Reno Kriz +1
The majority of work on summarization evaluation focuses on general summary quality (e.g., ROUGE, BERTScore) or specific desired properties (e.g., readability, factuality). However…
Findings of the MAGMaR 2026 Shared Task
Alexander Martin, Dengjia Zhang, Joel Brogan +7
This overview paper presents the results of the shared task for the second workshop on Multimodal Augmented Generation via Multimodal Retrieval (MAGMaR). In this shared task partic…
MARQUIS: A Three-Stage Pipeline for Video Retrieval-Augmented Generation
Debashish Chakraborty, Dengjia Zhang, Jialiang Jin +7
Retrieval-augmented generation from videos requires systems to retrieve relevant audiovisual evidence from large corpora and synthesize it into coherent, attributed text. Current a…
A Replicability Study of XTR
Rohan Jha, Reno Kriz, Benjamin Van Durme
The XTR (conteXtual Token Retrieval) algorithm is a modification to ColBERT retrieval that avoids the costly step of fully gathering and reranking the candidates' embeddings by imp…
Unified Multimodal Uncertain Inference
Dengjia Zhang, Alexander Martin, William Jurayj +3
We introduce Unified Multimodal Uncertain Inference (UMUI), a multimodal inference task spanning text, audio, and video, where models must produce calibrated probability estimates…
A Brief Comparison of Training-Free Multi-Vector Sequence Compression Methods
Rohan Jha, Chunsheng Zuo, Reno Kriz +1
While multi-vector retrieval models outperform single-vector models of comparable size in retrieval quality, their practicality is limited by substantially larger index sizes, driv…
Multi-Vector Index Compression in Any Modality
Hanxiang Qin, Alexander Martin, Rohan Jha +3
We study efficient multi-vector retrieval for late interaction in any modality. Late interaction has emerged as a dominant paradigm for information retrieval in text, images, visua…
RANKVIDEO: Reasoning Reranking for Text-to-Video Retrieval
Tyler Skow, Alexander Martin, Benjamin Van Durme +2
Reranking is a critical component of modern retrieval systems, which typically pair an efficient first-stage retriever with a more expressive model to refine results. While large r…
HLTCOE Evaluation Team at TREC 2025: VQA Track
Dengjia Zhang, Charles Weng, Katherine Guerrerio +5
The HLTCOE Evaluation team participated in TREC VQA's Answer Generation (AG) task, for which we developed a listwise learning framework that aims to improve semantic precision and…
Seeing Through the MiRAGE: Evaluating Multimodal Retrieval Augmented Generation
Alexander Martin, William Walden, Reno Kriz +5
We introduce MiRAGE, an evaluation framework for retrieval-augmented generation (RAG) from multimodal sources. As audiovisual media becomes a more prevalent source of information o…
Whisper-UT: A Unified Translation Framework for Speech and Text
Cihan Xiao, Matthew Wiesner, Debashish Chakraborty +7
Encoder-decoder models have achieved remarkable success in speech and text tasks, yet efficiently adapting these models to diverse uni/multi-modal scenarios remains an open challen…
WikiVideo: Article Generation from Multiple Videos
Alexander Martin, Reno Kriz, William Gantt Walden +5
We introduce the task of grounded article generation with the goal of creating a Wikipedia-style article from multiple diverse videos about real-world events -- from natural disast…
Video-ColBERT: Contextualized Late Interaction for Text-to-Video Retrieval
Arun Reddy, Alexander Martin, Eugene Yang +7
In this work, we tackle the problem of text-to-video retrieval (T2VR). Inspired by the success of late interaction techniques in text-document, text-image, and text-video retrieval…
MMMORRF: Multimodal Multilingual Modularized Reciprocal Rank Fusion
Saron Samuel, Dan DeGenaro, Jimena Guallar-Blasco +13
Videos inherently contain multiple modalities, including visual events, text overlays, sounds, and speech, all of which are important for retrieval. However, state-of-the-art multi…
Grounding Partially-Defined Events in Multimodal Data
Kate Sanders, Reno Kriz, David Etter +5
How are we able to learn about complex current events just from short snippets of video? While natural language enables straightforward ways to represent under-specified, partially…
MultiVENT 2.0: A Massive Multilingual Benchmark for Event-Centric Video Retrieval
Reno Kriz, Kate Sanders, David Etter +10
Efficiently retrieving and synthesizing information from large-scale multimodal collections has become a critical challenge. However, existing video retrieval datasets suffer from…
MultiVENT: Multilingual Videos of Events with Aligned Natural Text
Kate Sanders, David Etter, Reno Kriz +1
Everyday news coverage has shifted from traditional broadcasts towards a wide range of presentation formats such as first-hand, unedited video footage. Datasets that reflect the di…
On Event Individuation for Document-Level Information Extraction
William Gantt, Reno Kriz, Yunmo Chen +2
As information extraction (IE) systems have grown more adept at processing whole documents, the classic task of template filling has seen renewed interest as benchmark for document…
Ambiguous Images With Human Judgments for Robust Visual Event Classification
Kate Sanders, Reno Kriz, Anqi Liu +1
Contemporary vision benchmarks predominantly consider tasks on which humans can achieve near-perfect performance. However, humans are frequently presented with visual data that the…
GEMv2: Multilingual NLG Benchmarking in a Single Line of Code
Sebastian Gehrmann, Abhik Bhattacharjee, Abinaya Mahendiran +74
Evaluation in machine learning is usually informed by past choices, for example which datasets or metrics to use. This standardization enables the comparison on equal footing using…
Creating Multimedia Summaries Using Tweets and Videos
Anietie Andy, Siyi Liu, Daphne Ippolito +3
While popular televised events such as presidential debates or TV shows are airing, people provide commentary on them in real-time. In this paper, we propose a simple yet effective…