13 papers
BeyondSight: Object Permanence for End-to-End Autonomous Driving
Sandro Papais, Letian Wang, Mudit Jain +2
Autonomous driving operates in partially observable environments where actors may become fully occluded by other vehicles or infrastructure. Most end-to-end driving systems implici…
Video Generation Models are General-Purpose Vision Learners
Letian Wang, Chuhan Zhang, Rishabh Kabra +9
Driven by next-token prediction, NLP shifted from task-specific models into powerful generalist foundation models. What, then, is the equivalent catalyst needed to achieve a genera…
Grounded World Model for Semantically Generalizable Planning
Quanyi Li, Lan Feng, Haonan Zhang +4
In Model Predictive Control (MPC), world models predict the future outcomes of various action proposals, which are then scored to guide the selection of the optimal action. For vis…
LMGenDrive: Bridging Multimodal Understanding and Generative World Modeling for End-to-End Driving
Hao Shao, Letian Wang, Yang Zhou +5
Recent years have seen remarkable progress in autonomous driving, yet generalization to long-tail and open-world scenarios remains a major bottleneck for large-scale deployment. To…
DriveDreamer-Policy: A Geometry-Grounded World-Action Model for Unified Generation and Planning
Yang Zhou, Xiaofeng Wang, Hao Shao +8
Recently, world-action models (WAM) have emerged to bridge vision-language-action (VLA) models and world models, unifying their reasoning and instruction-following capabilities and…
THFM: A Unified Video Foundation Model for 4D Human Perception and Beyond
Letian Wang, Andrei Zanfir, Eduard Gabriel Bazavan +2
We present THFM, a unified video foundation model for human-centric perception that jointly addresses dense tasks (depth, normals, segmentation, dense pose) and sparse tasks (2d/3d…