38 papers
Vision-Language Grounding as Bidirectional Concept Correspondence
Jieyu Zhang, Ziqi Gao, Luke Zettlemoyer +1
Vision-language grounding connects language to visual content, yet most existing formulations reduce grounding to a unidirectional localization problem: given a prespecified text p…
Explain Before You Answer: A Survey on Compositional Visual Reasoning
Fucai Ke, Joy Hsu, Zhixi Cai +10
Compositional visual reasoning has emerged as a key research frontier in multimodal AI, aiming to endow machines with the human-like ability to decompose visual scenes, ground inte…
MolmoMotion: Forecasting Point Trajectories in 3D with Language Instruction
Jianing Zhang, Chenhao Zheng, Yajun Yang +10
Motion forecasting is central to visual intelligence: agents must anticipate how objects will move in order to plan actions, reason about physical interactions, and synthesize real…
Imaginative Perception Tokens Enhance Spatial Reasoning in Multimodal Language Models
Mahtab Bigverdi, Linjie Li, Weikai Huang +9
Vision language models (VLMs) excel at many tasks but still struggle with spatial reasoning when critical information is not directly observable. Many such problems require imagina…
TrajTok: Learning Trajectory Tokens enables better Video Understanding
Chenhao Zheng, Jieyu Zhang, Jianing Zhang +6
Tokenization in video models, typically through patchification, generates an excessive and redundant number of tokens. This severely limits video efficiency and scalability. While…
One Trajectory, One Token: Grounded Video Tokenization via Panoptic Sub-object Trajectory
Chenhao Zheng, Jieyu Zhang, Mohammadreza Salehi +5
Effective video tokenization is critical for scaling transformer models for long videos. Current approaches tokenize videos using space-time patches, leading to excessive tokens an…