From the 2 of 18 linked papers with an AI index.
18 papers
ReToken: One Token to Improve Vision-Language Models for Visual Retrieval
Yao Xiao, Reuben Tan, Zhen Zhu +3
ReToken introduces a single learnable embedding that acts as a retrieval token to select a sparse set of relevant visual tokens from a cached representation, improving vision-langu…
Building Agent Harnesses for Scientific Curation from Multimodal Sources
Sheng Zhang, Qin Liu, Renqian Luo +9
The paper introduces Beaver, an agent harness that extracts structured scientific information from papers by integrating text, tables, and figures while preserving provenance, and…
Diversity Over Frequency: Rethinking Tool Use in Visual Chain-of-Thought Agents
Dong-Hee Kim, Reuben Tan, Donghyun Kim
Visual agents employ external visual tools within visual chains of thought to incorporate fine-grained evidence. While prior work has mainly studied these tools in visual search ta…
Multimodal Reinforcement Learning with Adaptive Verifier for AI Agents
Reuben Tan, Baolin Peng, Zhengyuan Yang +16
Agentic reasoning models trained with multimodal reinforcement learning (MMRL) have become increasingly capable, yet they are almost universally optimized using sparse, outcome-bas…
InstrAct: Towards Action-Centric Understanding in Instructional Videos
Zhuoyi Yang, Jiapeng Yu, Reuben Tan +2
Understanding instructional videos requires recognizing fine-grained actions and modeling their temporal relations, which remains challenging for current Video Foundation Models (V…
AsgardBench -- Evaluating Visually Grounded Interactive Planning Under Minimal Feedback
Andrea Tupini, Lars Liden, Reuben Tan +2
With AsgardBench we aim to evaluate visually grounded, high-level action sequence generation and interactive planning, focusing specifically on plan adaptation during execution bas…