2.3k citations · 3.3k across the 91 of their papers we have counts for
110 papers
Interactive Visual Gym: Evaluating Visual Interaction Models with Simulated Users in Interactive Worlds
Chenhao Zheng, Zixian Ma, Kai Li +11
Visual interaction models that support users during ongoing physical tasks require evaluation in closed loop, where assistant responses can change subsequent user behavior and task…
EgoMo3R: Joint Egocentric Motion and Scene Reconstruction
Chaitanya Patel, Yusuke Kato, Rin Takahira +3
Understanding holistic 3D human motion and the surrounding environment from egocentric video is fundamental for applications in AR/VR and robotics. Although these tasks are inheren…
TRACER: A Multimodal Harness for Patient Care in Hospitals
Zane Durante, Paul Tang, Keshigeyan Chandrasegaran +11
Continuous video and clinical records provide complementary information for understanding how patient behavior evolves over a hospital stay. A central challenge is how to represent…
Evidence-Backed Video Question Answering
Shijie Wang, Honglu Zhou, Ziyang Wang +5
Current Video Large Language Models (Video LLMs) excel in question answering (QA) but largely operate as black boxes, providing textual answers without verifiable visual grounding.…
Artificial Intelligence Index Report 2026
Sha Sajadieh, Loredana Fattorini, Raymond Perrault +20
Welcome to the ninth edition of the AI Index report. As AI continues to advance rapidly, the question becomes whether the systems built around it can keep up. Governance frameworks…
Linear Scaling Video VLMs for Long Video Understanding
Cristobal Eyzaguirre, Jiajun Wu, Juan Carlos Niebles
Video vision-language models (VLMs) are increasingly used in long-horizon and streaming settings, yet most video encoders still rely on spatiotemporal self-attention, causing compu…
GPIC: A Giant Permissive Image Corpus for Visual Generation
Keshigeyan Chandrasegaran, Kyle Sargent, Suchir Agarwal +6
Studying scalable methods for visual generative modeling requires large, accessible, and stable datasets. We introduce GPIC, a Giant Permissive Image Corpus of approximately 28 tri…
SurfPhase: 3D Interfacial Dynamics in Two-Phase Flows from Sparse Videos
Yue Gao, Hong-Xing Yu, Sanghyeon Chang +5
Interfacial dynamics in two-phase flows govern momentum, heat, and mass transfer, yet remain difficult to measure experimentally. Classical techniques face intrinsic limitations ne…
Future Optical Flow Prediction Improves Robot Control & Video Generation
Kanchana Ranasinghe, Honglu Zhou, Yu Fang +7
Future motion representations, such as optical flow, offer immense value for control and generative tasks. However, forecasting generalizable spatially dense motion representations…
Robotic VLA Benefits from Joint Learning with Motion Image Diffusion
Yu Fang, Kanchana Ranasinghe, Le Xue +10
Vision-Language-Action (VLA) models have achieved remarkable progress in robotic manipulation by mapping multimodal observations and instructions directly to actions. However, they…
Active Video Perception: Iterative Evidence Seeking for Agentic Long Video Understanding
Ziyang Wang, Honglu Zhou, Shijie Wang +6
Long video understanding (LVU) is challenging because answering real-world queries often depends on sparse, temporally dispersed cues buried in hours of mostly redundant and irrele…
SMILE: A Composite Lexical-Semantic Metric for Question-Answering Evaluation
Shrikant Kendre, Austin Xu, Honglu Zhou +3
Traditional evaluation metrics for textual and visual question answering, like ROUGE, METEOR, and Exact Match (EM), focus heavily on n-gram based lexical similarity, often missing…
Strefer: Empowering Video LLMs with Space-Time Referring and Reasoning via Synthetic Instruction Data
Honglu Zhou, Xiangyu Peng, Shrikant Kendre +4
Next-generation AI companions must go beyond general video understanding to resolve spatial and temporal references in dynamic, real-world environments. Existing Video Large Langua…
UniEgoMotion: A Unified Model for Egocentric Motion Reconstruction, Forecasting, and Generation
Chaitanya Patel, Hiroki Nakamura, Yuta Kyuragi +3
Egocentric human motion generation and forecasting with scene-context is crucial for enhancing AR/VR experiences, improving human-robot interaction, advancing assistive technologie…
Taming generative video models for zero-shot optical flow extraction
Seungwoo Kim, Khai Loong Aw, Klemen Kotar +8
Extracting optical flow from videos remains a core computer vision problem. Motivated by the recent success of large general-purpose models, we ask whether frozen self-supervised v…
Exploring Diffusion Transformer Designs via Grafting
Keshigeyan Chandrasegaran, Michael Poli, Daniel Y. Fu +9
Designing model architectures requires decisions such as selecting operators (e.g., attention, convolution) and configurations (e.g., depth, width). However, evaluating the impact…
LAM SIMULATOR: Advancing Data Generation for Large Action Model Training via Online Exploration and Trajectory Feedback
Thai Hoang, Kung-Hsiang Huang, Shirley Kokane +12
Large Action Models (LAMs) for AI Agents offer incredible potential but face challenges due to the need for high-quality training data, especially for multi-steps tasks that involv…
Contra4: Evaluating Contrastive Cross-Modal Reasoning in Audio, Video, Image, and 3D
Artemis Panagopoulou, Le Xue, Honglu Zhou +6
Real-world decision-making often begins with identifying which modality contains the most relevant information for a given query. While recent multimodal models have made impressiv…
Understanding Complexity in VideoQA via Visual Program Generation
Cristobal Eyzaguirre, Igor Vasiljevic, Achal Dave +5
We propose a data-driven approach to analyzing query complexity in Video Question Answering (VideoQA). Previous efforts in benchmark design have relied on human expertise to design…
VLM Q-Learning: Aligning Vision-Language Models for Interactive Decision-Making
Jake Grigsby, Yuke Zhu, Michael Ryoo +1
Recent research looks to harness the general knowledge and reasoning of large language models (LLMs) into agents that accomplish user-specified goals in interactive environments. V…
AdaVid: Adaptive Video-Language Pretraining
Chaitanya Patel, Juan Carlos Niebles, Ehsan Adeli
Contrastive video-language pretraining has demonstrated great success in learning rich and robust video representations. However, deploying such video encoders on compute-constrain…
Artificial Intelligence Index Report 2025
Nestor Maslej, Loredana Fattorini, Raymond Perrault +20
Welcome to the eighth edition of the AI Index report. The 2025 Index is our most comprehensive to date and arrives at an important moment, as AI's influence across society, the eco…
APIGen-MT: Agentic Pipeline for Multi-Turn Data Generation via Simulated Agent-Human Interplay
Akshara Prabhakar, Zuxin Liu, Ming Zhu +12
Training effective AI agents for multi-turn interactions requires high-quality data that captures realistic human-agent dynamics, yet such data is scarce and expensive to collect m…
T*: Re-thinking Temporal Search for Long-Form Video Understanding
Jinhui Ye, Zihan Wang, Haosen Sun +9
Efficiently understanding long-form videos remains a significant challenge in computer vision. In this work, we revisit temporal search paradigms for long-form video understanding…